java - PIG/Hadoop issue: ERROR 2081: Unable to setup the load function -

java - PIG/Hadoop issue: ERROR 2081: Unable to setup the load function -

i'm running pig 0.13.0 , hadoop 2.5.1, both installed apache distros, they're not packages horton or cloudera or anything.

i'm working tutorial , can work fine when running pig locally ($> ./pig -x local), when trying run on hadoop instance error i'm having hard time researching on internet.

this command:

movies = load '/home/hduser/pig-tutorial-master/movies_data.csv' using pigstorage(',') (id,name,year,rating,duration); dump movies;

works fine running locally. when run in hadoop/mr mode, seems work fine when run first line of code:

grunt> movies = load '/home/hduser/pig-tutorial-master/movies_data.csv' using pigstorage(',') (id,name,year,rating,duration); 2014-10-29 18:16:26,281 [main] info org.apache.hadoop.conf.configuration.deprecation - fs.default.name deprecated. instead, utilize fs.defaultfs 2014-10-29 18:16:26,281 [main] info org.apache.hadoop.conf.configuration.deprecation - mapred.job.tracker deprecated. instead, utilize mapreduce.jobtracker.address

but when seek $> dump movies gives me trace:

grunt> dump movies 2014-10-29 18:17:15,419 [main] info org.apache.pig.tools.pigstats.scriptstate - pig features used in script: unknown 2014-10-29 18:17:15,420 [main] info org.apache.pig.newplan.logical.optimizer.logicalplanoptimizer - {rules_enabled=[addforeach, columnmapkeyprune, groupbyconstparallelsetter, limitoptimizer, loadtypecastinserter, mergefilter, mergeforeach, partitionfilteroptimizer, pushdownforeachflatten, pushupfilter, splitfilter, streamtypecastinserter], rules_disabled=[filterlogicexpressionsimplifier]} 2014-10-29 18:17:15,445 [main] warn org.apache.pig.data.schematuplebackend - schematuplebackend has been initialized 2014-10-29 18:17:15,469 [main] error org.apache.pig.tools.grunt.grunt - error 2081: unable setup load function. details @ logfile: /usr/local/pig/pig_1414606194436.log

the error 2081 i'm trying diagnose, can't find helps point me in right direction. ideas of start? assume it's hadoop installation , not pig, don't know. suggestions helpful.

thanks,

mark

edit: here total log output:

error 2081: unable setup load function. org.apache.pig.impl.logicallayer.frontendexception: error 1066: unable open iterator alias movies @ org.apache.pig.pigserver.openiterator(pigserver.java:912) @ org.apache.pig.tools.grunt.gruntparser.processdump(gruntparser.java:752) @ org.apache.pig.tools.pigscript.parser.pigscriptparser.parse(pigscriptparser.java:372) @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:228) @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:203) @ org.apache.pig.tools.grunt.grunt.run(grunt.java:66) @ org.apache.pig.main.run(main.java:542) @ org.apache.pig.main.main(main.java:156) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:57) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43) @ java.lang.reflect.method.invoke(method.java:606) @ org.apache.hadoop.util.runjar.main(runjar.java:212) caused by: org.apache.pig.pigexception: error 1002: unable store alias movies @ org.apache.pig.pigserver.storeex(pigserver.java:1015) @ org.apache.pig.pigserver.store(pigserver.java:974) @ org.apache.pig.pigserver.openiterator(pigserver.java:887) ... 12 more caused by: org.apache.pig.backend.executionengine.execexception: error 0: exception while executing (name: movies: store(hdfs://localhost:54310/tmp/temp-1276361014/tmp-2000190966:org.apache.pig.impl.io.interstorage) - scope-1 operator key: scope-1): org.apache.pig.backend.executionengine.execexception: error 2081: unable setup load function. @ org.apache.pig.backend.hadoop.executionengine.physicallayer.physicaloperator.processinput(physicaloperator.java:289) @ org.apache.pig.backend.hadoop.executionengine.physicallayer.relationaloperators.postore.getnexttuple(postore.java:143) @ org.apache.pig.backend.hadoop.executionengine.fetch.fetchlauncher.runpipeline(fetchlauncher.java:160) @ org.apache.pig.backend.hadoop.executionengine.fetch.fetchlauncher.launchpig(fetchlauncher.java:81) @ org.apache.pig.backend.hadoop.executionengine.hexecutionengine.launchpig(hexecutionengine.java:275) @ org.apache.pig.pigserver.launchplan(pigserver.java:1367) @ org.apache.pig.pigserver.executecompiledlogicalplan(pigserver.java:1352) @ org.apache.pig.pigserver.storeex(pigserver.java:1011) ... 14 more caused by: org.apache.pig.backend.executionengine.execexception: error 2081: unable setup load function. @ org.apache.pig.backend.hadoop.executionengine.physicallayer.relationaloperators.poload.getnexttuple(poload.java:127) @ org.apache.pig.backend.hadoop.executionengine.physicallayer.physicaloperator.processinput(physicaloperator.java:281) ... 21 more caused by: org.apache.hadoop.mapreduce.lib.input.invalidinputexception: input path not exist: hdfs://localhost:54310/home/hduser/pig-tutorial-master/movies_data.csv @ org.apache.hadoop.mapreduce.lib.input.fileinputformat.singlethreadedliststatus(fileinputformat.java:321) @ org.apache.hadoop.mapreduce.lib.input.fileinputformat.liststatus(fileinputformat.java:264) @ org.apache.pig.backend.hadoop.executionengine.mapreducelayer.pigtextinputformat.liststatus(pigtextinputformat.java:36) @ org.apache.hadoop.mapreduce.lib.input.fileinputformat.getsplits(fileinputformat.java:385) @ org.apache.pig.impl.io.readtoendloader.init(readtoendloader.java:190) @ org.apache.pig.impl.io.readtoendloader.<init>(readtoendloader.java:146) @ org.apache.pig.backend.hadoop.executionengine.physicallayer.relationaloperators.poload.setup(poload.java:95) @ org.apache.pig.backend.hadoop.executionengine.physicallayer.relationaloperators.poload.getnexttuple(poload.java:123) ... 22 more ================================================================================

after searching solution error 2081, started looking @ errors in log file more closely. issue of trying access local files mr mode. hadn't noticed in documentation how access info in mr vs. local, issue.

if running in mr, must access files via hdfs://hostname:54310. locally can access them path.

this s.o. question solution: how load files on hadoop cluster using apache pig?.

java apache hadoop apache-pig