NoSuchMethodError while using joda-time-2.2.jar in PIG

Hi Guys,

I spent many hours solving this before I found the solution.

In the UDF we are using some APIs of Joda-time. The issue is while running the job it fails. There are no compilation issues in eclipse.  In eclipse also you might face this issue because the pig-version.jar also has joda package. Just put the joda-time-version.jar first in the classpath (before pig.jar) and you’r issue will be fixed.

I was trying to run it on the cygwin on windows machine but same issue can be seen on the linux box also.

Common stacktrace you might see :

2013-08-11 13:01:06,911 [Thread-9] WARN org.apache.hadoop.mapred.LocalJobRunner – job_local_0001

java.lang.NoSuchMethodError: org.joda.time.DateTime.now(Lorg/joda/time/DateTimeZone;)Lorg/joda/time/DateTime;
at com.myproject.pig.udf.ExtractDataByDates.exec(ExtractDataByDates.java:178)
at com.myproject.pig.udf.ExtractDataByDates.exec(ExtractDataByDates.java:12)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:337)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:381)

I tried many options :

1) Registering this joda jar in the PIG script using REGISTER call. (Didn’t work)

2) using -Dpig.additional.jars=/path/to/joda-time/jar (Did not work)

3) Set this jar in $HADOOP_CLASSPATH (Did not work)

4) Set this jar in the $classpath (Did not work)

5) Set the jar in the $PIG_CLASSPATH (It works)

export PIG_CLASSPATH=$PIG_CLASSPATH:/path/to/joda-time-2.2.jar

Advertisements

Tagged: , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: