开发者社区> 问答> 正文

运行Apache Flink时出现java.lang.ClassNotFoundException:com.mongodb.hadoop.mapred.MongoInputFormat

我正在用Java创建一个flink系统来读写MongoDB。我在JDK 1.8.0_181中使用mongo-hadoop-core版本1.3.2和Apache flink 1.6。

当我在Eclipse中运行源代码时,一切都运行成功,但是当我使用flink命令运行JAR文件时,我总是会收到此错误。

这是我的pom.xml

http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0
org.apache.flink
test-mongodb-2
1.0-SNAPSHOT

<plugins>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
            <execution>
                <phase>package</phase>
                <goals>
                    <goal>shade</goal>
                </goals>
                <configuration>
                    <artifactSet>
                        <excludes>
                            <exclude>com.google.code.findbugs:jsr305</exclude>
                            <exclude>org.slf4j:*</exclude>
                            <exclude>log4j:*</exclude>
                        </excludes>
                    </artifactSet>
                    <filters>
                        <filter>
                            <!-- Do not copy the signatures in the META-INF folder.
                            Otherwise, this might cause SecurityExceptions when using the JAR. -->
                            <artifact>*:*</artifact>
                            <excludes>
                                <exclude>META-INF/*.SF</exclude>
                                <exclude>META-INF/*.DSA</exclude>
                                <exclude>META-INF/*.RSA</exclude>
                            </excludes>
                        </filter>
                    </filters>
                    <transformers>
                        <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                            <mainClass>my.programs.main.clazz</mainClass>
                        </transformer>
                    </transformers>
                </configuration>
            </execution>
        </executions>
    </plugin>
</plugins>

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-java</artifactId>
  <version>1.6.0</version>
  <scope>provided</scope>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-streaming-java_2.11</artifactId>
  <version>1.6.0</version>
  <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-core</artifactId>
    <version>1.6.0</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-clients_2.11</artifactId>
  <version>1.6.0</version>
</dependency>
<dependency>
    <groupId>org.mongodb.mongo-hadoop</groupId>
    <artifactId>mongo-hadoop-core</artifactId>
    <version>1.3.2</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-hadoop-compatibility_2.11</artifactId>
    <version>1.6.0</version>
</dependency>



但是在我将mongo-hadoop-core版本更改为1.3.2后,它会产生不同的错误

java.lang.RuntimeException: Could not look up the main(String[]) method from the class MongoDBExample: org/apache/flink/api/java/hadoop/mapred/HadoopInputFormat
at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:499)
at org.apache.flink.client.program.PackagedProgram.(PackagedProgram.java:218)
at org.apache.flink.client.program.PackagedProgram.(PackagedProgram.java:128)
at org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:856)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:206)
at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1044)
at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1120)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1120)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/api/java/hadoop/mapred/HadoopInputFormat
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:493)
... 11 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.api.java.hadoop.mapred.HadoopInputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 17 more
到底哪里出错了?

展开
收起
flink小助手 2018-12-10 13:19:33 14449 0
1 条回答
写回答
取消 提交回答
  • flink小助手会定期更新直播回顾等资料和文章干货,还整合了大家在钉群提出的有关flink的问题及回答。

    我在pom.xml中添加了mongo-hadoop-core-2.0.2.jar并在其中找到了类。请尝试CNTR + SHIFT + T并检查类路径中是否存在该类

    2019-07-17 23:19:12
    赞同 展开评论 打赏
问答排行榜
最热
最新

相关电子书

更多
MongoDB多数据中心的方案选型之路 立即下载
阿里云MongoDB云服务构建 立即下载
饿了么高级架构师陈东明:MongoDB是如何逐步提高可靠性的 立即下载

相关镜像