Erlang garbage collection -

Erlang garbage collection -

i need help in investigation of issue erlang memory consumption. how typical, isn't it?

we have 2 different deployment schemes.

in first scheme running many identical nodes on little virtual machines (in amazon aws), 1 node per machine. each machine has 4gb of ram. in deployment scheme running nodes on big baremetal machines (with 64 gb of ram), many nodes per machine. in deployment nodes isolated in docker containers (with memory limit set 4 gb).

i've noticed, heap of processes in dockerized nodes hogging 3 times much more ram, heaps in non-dockerized nodes identical load. suspect garbage collection in non-dockerized nodes more aggressive. unfortunately, don't have garbage collection statistics, obtain asap.

to give more information, should using hipe r17.1 on ubuntu 14.04 stock kernel. in both schemes running 8 schedulers per node, , using default fullsweep_after flag.

my blind suggestion erlang default garbage collection relies (somehow) on /proc/meminfo (which not actual in dockerized environment). not c-guy , not familiar emulator internals, point me places in erlang sources responsible garbage collection , emulator options can utilize tweak behavior?

first thing know, garbage collection erlang process based. each process gc in own time, , independently each other. garbage collection in scheme dependent on info in processes, not operating scheme itself.

that said, there differencess between memory consumption eralang point of view, , scheme point of view. why comparing erlang:memory scheme saying thought (it show binary leaks, or other memory problems).

if understand little more erlang internals recommend 2 talks:

https://www.youtube.com/watch?v=qbzh0l_0pxi

https://www.youtube.com/watch?v=yupax11vzyi

and little improve debugging of memory management reccomend starting http://ferd.github.io/recon/

garbage-collection erlang docker