Wednesday 15 February 2012

java - How to simply set up a serializer for storm Bolt/Spout members? -



java - How to simply set up a serializer for storm Bolt/Spout members? -

i have bolt uses mill interface create tools uses. creates tools when prepare called. mill implementation has basic members (strings, integers) should serializable default.

when run topology notserializableexception, coming mill implementation. wondering how register serializer factory.

here's example:

public class demo extends baserichbolt { public static interface iexecutor { public void execute( tuple tuple ); } public static interface iexecutorcreator { public iexecutor create( map map, topologycontext tc, outputcollector oc ); }; public static class dummyexecutor implements iexecutor { public void execute( tuple tuple ) {} }; public static class dummyexecutorcreator implements iexecutorcreator { string name_; public dummyexecutorcreator(string name) { this.name_ = name;} public iexecutor create(map map, topologycontext tc, outputcollector oc) { homecoming new dummyexecutor(); } }; public void declareoutputfields(outputfieldsdeclarer ofd) { } private iexecutor executor_; private iexecutorcreator creator_; public demo(iexecutorcreator creator) { this.creator_ = creator; } public void prepare(map map, topologycontext tc, outputcollector oc) { this.executor_ = this.creator_.create(map, tc, oc); } public void execute(tuple tuple) { this.executor_.execute(tuple); } }

and error when seek run in topology: java.io.notserializableexception: demo$dummyexecutorcreator

as side note i'm starting wonder, why don't storm have register factories instead of bolts , spouts. since in end serialized , copied across different threads, it'd improve give storm mean generate bolts , separate concerns.

don't set values in constructor. constructor called when topology first built driver , before topology submitted cluster. that's why instance variables set, whether class initialization or in constructor, must serializable.

instead, leave instance variables null , set them in prepare() method. when won't need serialize values, works non-serializable instance variables, too.

java serialization apache-storm

No comments:

Post a Comment