Monday, March 25, 2013

Java heap space or GC out of limit issue


set hive.map.aggr=true;
set hive.map.aggr.hash.force.flush.memory.threshold=0.75;
set hive.map.aggr.hash.percentmemory=0.3;
set hive.groupby.mapaggr.checkinterval=10000;
set mapred.child.java.opts=-Xmx3072M;
set hive.exec.compress.output=true;
set io.seqfile.compression.type=BLOCK;
all of those are good param
except maybe set hive.exec.compress.output=true;
and
set io.seqfile.compression.type=BLOCK;

No comments:

Post a Comment