https://github.com/apache/flink/tree/blink
开源了,目前是 flink 的一个分支
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.myorg.quickstart</groupId> <artifactId>quickstart</artifactId> <version>0.1</version> <packaging>jar</packaging> <name>Flink Quickstart Job</name> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <flink.version>1.10.1</flink.version> <java.version>1.8</java.version> <scala.binary.version>2.11</scala.binary.version> <maven.compiler.source>${java.version}</maven.compiler.source> <maven.compiler.target>${java.version}</maven.compiler.target> </properties> <repositories> <repository> <id>apache.snapshots</id> <name>Apache Development Snapshot Repository</name> <url>https://repository.apache.org/content/repositories/snapshots/</url> <releases> <enabled>false</enabled> </releases> <snapshots> <enabled>true</enabled> </snapshots> </repository> </repositories> <dependencies> <!-- Apache Flink dependencies --> <!-- These dependencies are provided, because they should not be packaged into the JAR file. --> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>${flink.version}</version> <scope>provided</scope> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_${scala.binary.version}</artifactId> <version>${flink.version}</version> <scope>provided</scope> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> <version>1.7.7</version> <scope>runtime</scope> </dependency> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.17</version> <scope>runtime</scope> </dependency> </dependencies> <build> <plugins> <!-- Java Compiler --> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.1</version> <configuration> <source>${java.version}</source> <target>${java.version}</target> </configuration> </plugin> <!-- We use the maven-shade plugin to create a fat jar that contains all necessary dependencies. --> <!-- Change the value of <mainClass>...</mainClass> if your program entry point changes. --> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>3.1.1</version> <executions> <!-- Run shade goal on package phase --> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <artifactSet> <excludes> <exclude>org.apache.flink:force-shading</exclude> <exclude>com.google.code.findbugs:jsr305</exclude> <exclude>org.slf4j:*</exclude> <exclude>log4j:*</exclude> </excludes> </artifactSet> <filters> <filter> <!-- Do not copy the signatures in the META-INF folder. Otherwise, this might cause SecurityExceptions when using the JAR. --> <artifact>*:*</artifact> <excludes> <exclude>META-INF/*.SF</exclude> <exclude>META-INF/*.DSA</exclude> <exclude>META-INF/*.RSA</exclude> </excludes> </filter> </filters> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> <mainClass>org.myorg.quickstart.StreamingJob</mainClass> </transformer> </transformers> </configuration> </execution> </executions> </plugin> </plugins> <pluginManagement> <plugins> <!-- This improves the out-of-the-box experience in Eclipse by resolving some warnings. --> <plugin> <groupId>org.eclipse.m2e</groupId> <artifactId>lifecycle-mapping</artifactId> <version>1.0.0</version> <configuration> <lifecycleMappingMetadata> <pluginExecutions> <pluginExecution> <pluginExecutionFilter> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <versionRange>[3.1.1,)</versionRange> <goals> <goal>shade</goal> </goals> </pluginExecutionFilter> <action> <ignore/> </action> </pluginExecution> <pluginExecution> <pluginExecutionFilter> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <versionRange>[3.1,)</versionRange> <goals> <goal>testCompile</goal> <goal>compile</goal> </goals> </pluginExecutionFilter> <action> <ignore/> </action> </pluginExecution> </pluginExecutions> </lifecycleMappingMetadata> </configuration> </plugin> </plugins> </pluginManagement> </build> </project>
不会,大表少则几十G,大的上T的数据,内存有怎么扛得住?
我认为,应该是比直接用算子要慢一些吧。
添加 flink-scala flink-stream-scala
17-18,詹姆斯-哈登,72,8.8,1.8,0.7,30.4
16-17,拉塞尔-维斯布鲁克,81,10.4,1.6,0.4,31.6
15-16,斯蒂芬-库里,79,6.7,2.1,0.2,30.1
14-15,拉塞尔-维斯布鲁克,67,8.6,2.1,0.2,28.1
13-14,凯文-杜兰特,81,5.5,1.3,0.7,32
12-13,卡梅罗-安东尼,67,2.6,0.8,0.5,28.7
11-12,凯文-杜兰特,66,3.5,1.3,1.2,28
10-11,凯文-杜兰特,78,2.7,1.1,1,27.7
09-10,凯文-杜兰特,82,2.8,1.4,1,30.1
08-09,德维恩-韦德,79,7.5,2.2,1.3,30.2
07-08,勒布朗-詹姆斯,75,7.2,1.8,1.1,30
06-07,科比-布莱恩特,77,5.4,1.4,0.5,31.6
05-06,科比-布莱恩特,80,4.5,1.8,0.4,35.4
04-05,阿仑-艾佛森,75,7.9,2.4,0.1,30.7
03-04,特雷西-麦克格雷迪,67,5.5,1.4,0.6,28
02-03,特雷西-麦克格雷迪,75,5.5,1.7,0.8,32.1
01-02,阿仑-艾佛森,60,5.5,2.8,0.2,31.4
00-01,阿仑-艾佛森,71,4.6,2.5,0.3,31.1
99-00,沙奎尔-奥尼尔,79,3.8,0.5,3,29.7
98-99,阿仑-艾佛森,48,4.6,2.3,0.1,26.8
97-98,迈克尔-乔丹,82,3.5,1.7,0.5,28.7
96-97,迈克尔-乔丹,82,4.3,1.7,0.5,29.6
95-96,迈克尔-乔丹,82,4.3,2.2,0.5,30.4
94-95,沙奎尔-奥尼尔,79,2.7,0.9,2.4,29.3
93-94,大卫-罗宾逊,80,4.8,1.7,3.3,29.8
92-93,迈克尔-乔丹,78,5.5,2.8,0.8,32.6
91-92,迈克尔-乔丹,80,6.1,2.3,0.9,30.1
90-91,迈克尔-乔丹,82,5.5,2.7,1,31.5
89-90,迈克尔-乔丹,82,6.3,2.8,0.7,33.6
88-89,迈克尔-乔丹,81,8,2.9,0.8,32.5
87-88,迈克尔-乔丹,82,5.9,3.2,1.6,35
86-87,迈克尔-乔丹,82,4.6,2.9,1.5,37.1
85-86,多米尼克-威尔金斯,78,2.6,1.8,0.6,30.3
84-85,伯纳德-金,55,3.7,1.3,0.3,32.9
83-84,阿德里安-丹特利,79,3.9,0.8,0.1,30.6
82-83,阿历克斯-英格利什,82,4.8,1.4,1.5,28.4
81-82,乔治-格文,79,2.4,1,0.6,32.3
import org.apache.flink.table.api.*; import static org.apache.flink.table.api.Expressions.*; EnvironmentSettings settings = EnvironmentSettings.newInstance().inStreamingMode().build(); TableEnvironment tEnv = TableEnvironment.create(env);
不用类封装,如果是Java开发的话用Tuple2就好了,如果是用scala这个都不需要,直接链式编程
百度一下,值得拥有
是的,本机localhost
直接maven 项目不就可以了么
在这没有任何意义,就是给你演示一下。执行个算子给你看看而已。为了代表实际生产中的一些了转换
这个类是scala版本的批处理api包,你需要引入依赖flink-scala_2.11