MapReduce 生成Hfile 的Mrunit单元测试提示Serialization空指针

 

使用Mrunit对生成Hfile 的程序进行单元测试的时候发现提示 Serialization的  NullPointerException 跟踪代码后发现se 初始化后是null
提示信息:
    
    
  1. java.lang.NullPointerException
  2. at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:61)
  3. at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:81)
原因无法初始化  serializer
   
   
  1. try {
  2. serializer = (Serializer<Object>) serializationFactory //这里为空
  3. .getSerializer(clazz);
  4. deserializer = (Deserializer<Object>) serializationFactory
  5. .getDeserializer(clazz);
  6. } catch (NullPointerException e) {
  7. throw new IllegalStateException(
  8. "No applicable class implementing Serialization in conf at io.serializations for "
  9. + orig.getClass(), e);
  10. }
  11. try {
  12. final DataOutputBuffer outputBuffer = new DataOutputBuffer();
  13. serializer.open(outputBuffer);
  14. serializer.serialize(orig);
  15. final DataInputBuffer inputBuffer = new DataInputBuffer();
  16. inputBuffer.reset(outputBuffer.getData(), outputBuffer.getLength());
  17. deserializer.open(inputBuffer);
  18. return (T) deserializer.deserialize(copy);
  19. } catch (final IOException e) {
  20. throw new RuntimeException(e);
  21. }
需要设置:
   
   
  1. Configuration conf = mapDriver.getConfiguration();
  2. conf.setStrings("io.serializations", conf.get("io.serializations"),
  3. MutationSerialization.class.getName(),
  4. ResultSerialization.class.getName(),
  5. KeyValueSerialization.class.getName());
使用Mrunit对生成Hfile 的程序进行单元测试的时候发现提示 Serialization的  NullPointerException 跟踪代码后发现se 初始化后是null
提示信息:
     
     
  1. java.lang.NullPointerException
  2. at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:61)
  3. at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:81)
原因无法初始化  serializer
    
    
  1. try {
  2. serializer = (Serializer<Object>) serializationFactory //这里为空
  3. .getSerializer(clazz);
  4. deserializer = (Deserializer<Object>) serializationFactory
  5. .getDeserializer(clazz);
  6. } catch (NullPointerException e) {
  7. throw new IllegalStateException(
  8. "No applicable class implementing Serialization in conf at io.serializations for "
  9. + orig.getClass(), e);
  10. }
  11. try {
  12. final DataOutputBuffer outputBuffer = new DataOutputBuffer();
  13. serializer.open(outputBuffer);
  14. serializer.serialize(orig);
  15. final DataInputBuffer inputBuffer = new DataInputBuffer();
  16. inputBuffer.reset(outputBuffer.getData(), outputBuffer.getLength());
  17. deserializer.open(inputBuffer);
  18. return (T) deserializer.deserialize(copy);
  19. } catch (final IOException e) {
  20. throw new RuntimeException(e);
  21. }
需要设置:
    
    
  1. Configuration conf = mapDriver.getConfiguration();
  2. conf.setStrings("io.serializations", conf.get("io.serializations"),
  3. MutationSerialization.class.getName(),
  4. ResultSerialization.class.getName(),
  5. KeyValueSerialization.class.getName());
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值