Mike had a question about passing a small amount of data to the mapper/reducer (e.g., config info, constants, etc.).
Here how you do it:
JobConf conf = new JobConf(Foo.class); ... conf.set("property1", "string"); conf.setInt("property2", 2);
JobConf extends Configuration, and these methods are defined in Configuration.
And inside your mapper:
private static class MyMapper extends MapReduceBase implements... { public void configure(JobConf job) { String s = job.get("property1"); int value = job.get("property2"); ... } ... }
The configure method is called in your mapper upon initialization (and similarly for reducer).
1 comment:
Mike had a question about passing a small amount of data to the mapper/reducer (e.g., config info, constants, etc.).
Here how you do it:
JobConf conf = new JobConf(Foo.class);
...
conf.set("property1", "string");
conf.setInt("property2", 2);
JobConf extends Configuration, and these methods are defined in Configuration.
And inside your mapper:
private static class MyMapper extends MapReduceBase implements... {
public void configure(JobConf job) {
String s = job.get("property1");
int value = job.get("property2");
...
}
...
}
The configure method is called in your mapper upon initialization (and similarly for reducer).
Hope this helps!
Post a Comment