Friday 10 January 2020

Prevent node from running out of memory in a docker build that a 600 MB memory limit

I am running docker build with a limit on the build's memory and CPU. To stay within the build's CPU and memory limits, I am also limiting Node to a heap size of 325 MB. This is the docker build command.

docker build --build-arg NODE_OPTIONS=--max-old-space-size=325 \
             --memory=600m --memory-swap=-1 \ 
             --cpu-period=100000 --cpu-quota=50000 \
             --no-cache --tag farm_app_image:latest --file Dockerfile .

Build Resource Summary

  • Node JS Heap Limit: 325 MB
  • Build Memory: 600 MB with unlimited use of swap files.
  • Build CPU Time: 50%

Despite having a Node heap limit that is below the build memory, and despite having unlimited swap, the npm run build runs out of memory on the react-scripts build step.

Error Ouput

  > react-scripts build                            

  Creating an optimized production build...                                                                                                                                          
EXEC : FATAL error : Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory [/app/FarmLandLasanga/FarmLandLasanga.csproj]                       

  <--- Last few GCs --->                                                                                                                                                             

  [186:0x5e6ca40]   143688 ms: Mark-sweep 319.3 (329.2) -> 319.2 (329.9) MB, 1278.3 / 0.0 ms  (average mu = 0.118, current mu = 0.002) allocation failure scavenge might not succeed 
  [186:0x5e6ca40]   145181 ms: Mark-sweep 320.3 (329.9) -> 320.2 (331.2) MB, 1488.0 / 0.1 ms  (average mu = 0.060, current mu = 0.003) allocation failure scavenge might not succeed 


  <--- JS stacktrace --->                                                                                                                                                            

  ==== JS stack trace =========================================                                                                                                                      

      0: ExitFrame [pc: 0x1391439]                                                                                                                                                   
      1: StubFrame [pc: 0x1316d29]                                                                                                                                                   
  Security context: 0x154b83ac08d1 <JSObject>                                                                                                                                        
      2: /* anonymous */ [0x14bbf66e0a09] [/app/FarmLandLasanga/ClientAppTypeScript/node_modules/webpack-sources/node_modules/source-map/lib/source-node.js:~86] [pc=0xa4979a73d50](t
his=0x1e34cc96a351 <JSFunction SourceNode (sfi = 0xd8bc3128ee1)>,0x307dd6e664f1 <Object map = 0xb3550d64299>)                                                                        
      3: arguments adaptor frame: 3->1                                                                                                                                               
      4:...                                                                                                                                                                          


  Writing Node.js report to file: report.20191116.195427.186.0.001.json                                                                                                              
  Node.js report completed                                                                                                                                                           
   1: 0x9e9f40 node::Abort() [node]                                                                                                                                                  
   2: 0x9ec192 node::OnFatalError(char const*, char const*) [node]                                                                                                                   
   3: 0xb4611e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]                                                                                         
   4: 0xb46499 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]                                                                           
   5: 0xcf3535  [node]                                                                                                                                                               
   6: 0xcf3bc6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]                                                                                            
   7: 0xd003fa v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]                                                              
   8: 0xd01305 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]                                  
   9: 0xd03dac v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]           
  10: 0xcca66b v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]                                                
  11: 0x100eb9e v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]                                                                  
  12: 0x1391439  [node]                                                                                                                                                              
  npm ERR! code ELIFECYCLE                                                                                                                                                           
  npm ERR! errno 1                                                                                                                                                                   
  npm ERR! farm-land-lasagna@0.1.0 build: `react-scripts --max_old_space_size=325 build`                                                                                             
  npm ERR! Exit status 1                                                                                                                                                             
  npm ERR!                                                                                                                                                                           
  npm ERR! Failed at the farm-land-lasagna@0.1.0 build script.                                                                                                                       
  npm ERR! This is probably not a problem with npm. There is likely additional logging output above.                                                                                 

  npm ERR! A complete log of this run can be found in:                                                                                                                               
  npm ERR!     /root/.npm/_logs/2019-11-16T19_54_27_392Z-debug.log                                                                                                                   

Question

What other changes, if any, can I make to the docker build command so that it succeeds with a 600 MB limit?

References

Additional Attempts

  • With --memory=900m the build succeeds.
  • react-scripts --max_old_space_size=325 build inside of Docker fails.
  • react-scripts --max_old_space_size=325 build outside of Docker succeeds.


from Prevent node from running out of memory in a docker build that a 600 MB memory limit

No comments:

Post a Comment