Avoid heap error while looping in coldfusion

Question

I am trying to loop over a 2-D array whose dimensions are 12000 * 20 and I am continuously getting java.lang.OutOfMemoryError.

Initially I thought this might be because of heap size so I increased my heap size but I was still getting the same error. So I ran a garbage collector like this:

<cflock name="checkMemory" type="exclusive" timeout="1" throwontimeout="yes">
    <cfset objSystem = CreateObject( "java", "java.lang.System" )>
    <cfset objSystem.gc()>
</cflock>

and I dumped the free memory which was around 850MB:

<cfset runtime = CreateObject("java","java.lang.Runtime").getRuntime()>
<cfset freeMemory = runtime.freeMemory()>
<cfdump var="#freeMemory#" label="free">

Here I am trying to create a XML variable and while looping I am getting heap error:

<cfxml variable="variables.XML">
  <cfoutput>
    <ROWS>
      <cfloop from="3" to="#arrayLen(local.array)#" index="i" step="1">
        <ROW>
          <cfloop from="1" to="#arrayLen(local.array[2])#" index="j" step="1">
            <#ucase(local.array[2][j])#>
              <![CDATA[#trim(local.array[i][j])#]]>
            </#ucase(local.array[2][j])#>
          </cfloop>
        </ROW>
      </cfloop>
    </ROWS>
  </cfoutput>
</cfxml>

This is the stack trace:

java.lang.OutOfMemoryError at java.io.WinNTFileSystem.getBooleanAttributes(Native Method) at java.io.File.exists(File.java:733) at coldfusion.xml.XmlProcessor.getSourceURL(XmlProcessor.java:246) at coldfusion.xml.XmlProcessor.parse(XmlProcessor.java:155) at coldfusion.tagext.lang.XmlTag.doEndTag(XmlTag.java:85) at cffeeds2ecfc1003675922$funcDEMO1._factor8(C:\component\abc.cfc:1235) at cffeeds2ecfc1003675922$funcDEMO1.runFunction(C:\component\abc.cfc:1192) at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472) at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405) at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368) at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55) at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321) at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:220) at coldfusion.runtime.CfJspPage._invokeUDF(CfJspPage.java:2582) at cffeeds2ecfc1003675922$funcDEMO.runFunction(\component\abc.cfc:935) at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472) at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405) at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368) at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55) at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321) at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:517) at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:496) at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:355) at coldfusion.filter.ComponentFilter.invoke(ComponentFilter.java:188) at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:374) at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48) at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40) at coldfusion.filter.PathFilter.invoke(PathFilter.java:94) at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70) at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28) at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38) at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46) at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38) at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22) at coldfusion.xml.rpc.CFCServlet.invoke(CFCServlet.java:139) at coldfusion.xml.rpc.CFCServlet.doPost(CFCServlet.java:290) at javax.servlet.http.HttpServlet.service(HttpServlet.java:760) at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327) at javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89) at jrun.servlet.FilterChain.doFilter(FilterChain.java:86) at com.intergral.fusionreactor.filter.FusionReactorFilter.i(FusionReactorFilter.java:566) at com.intergral.fusionreactor.filter.FusionReactorFilter.c(FusionReactorFilter.java:258) at com.intergral.fusionreactor.filter.FusionReactorFilter.doFilter(FusionReactorFilter.java:164) at jrun.servlet.FilterChain.doFilter(FilterChain.java:94) at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42) at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46) at jrun.servlet.FilterChain.doFilter(FilterChain.java:94) at jrun.servlet.FilterChain.service(FilterChain.java:101) at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106) at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42) at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286) at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543) at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203) at jrunx.scheduler.ThreadPool$DownstreamMetrics.invokeRunnable(ThreadPool.java:320) at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428) at jrunx.scheduler.ThreadPool$UpstreamMetrics.invokeRunnable(ThreadPool.java:266) at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

Is there any better solution to avoid looping like this or to resolve this error?

I created a cfm page locally and added some variables in it like this:

<cfset runtime = CreateObject("java","java.lang.Runtime").getRuntime()>
<cfset freeMemory = runtime.freeMemory()>
<cfset totalMemory = runtime.totalMemory()>
<cfset maxMemory = runtime.maxMemory()>
<cfdump var="#freeMemory#" label="free">
<cfdump var="#totalMemory#" label="total">
<cfdump var="#maxMemory#" label="max">

Every time I was refreshing this page the free memory size was decreasing until I ran the GC. I am still trying to figure out why this is happening. Any suggestions in this regard?

Please help. Thanks in advance.

The relevant part of the code here would be the bit that you abbreviate as //Some operation. It's impossible to comment what one can do to make that code more memory-usage friendly without seeing it. — Adam Cameron
So array2 isn't actually the sub-array of array, it's just a different area of the same length, defining all the XML nodes? It might help to show how you create your array variables to confirm this. I'd be tempted to just use UCase when you populate array2, thus reducing 12000 * 20 * 2 function calls. — duncan
I'd perhaps build the string first, then convert it to XML, rather than use CFXML, which I've had memory issues with in the past. What are you doing with this XML once you've created it? Also agree with Duncan: you don't need to ucase() the same string multiple times. Possibly also use StringBuilder, rather than String for this. — Adam Cameron
What happens if you move the CFOUTPUT to the outside of your loops so you aren't calling it 240,00 separate times. When building large strings, I prefer to dump string fragments into a 1 dimensional array & then use ArrayToList(Array, "") at the end. Then perform CFXML with the final, single generated string versus doing it all within the CFXML block. — James Moberg

Beginner Beginner · Accepted Answer · 2015-03-26T08:03:53

After suggestions from Adam Cameron, Dan Bracuk and James Moberg, I have done the following and I am not getting the heap error now. I suspect that it was because of CFXML tag but not sure.

Converting all element of array[2] to uppercase:

<cfloop from="1" to="#arrayLen(local.array[2])#" index="i" step="1">
  <cfset local.array[2][i] = ucase(local.array[2][i])>
</cfloop>

Then using cfsavecontent to generate the xml string:

<cfsavecontent variable="local.xmlString">
  <?xml version="1.0" encoding="UTF-8"?>
  <ROWS>
    <cfoutput>
      <cfloop from="3" to="#arrayLen(local.array)#" index="local.currentRow" step="1">
        <ROW>
          <cfloop from="1" to="#arrayLen(local.array[2])#" index="local.currentColumn" step="1">
            <#local.array[2][local.currentColumn]#>
              <![CDATA[#trim(local.array[local.currentRow][local.currentColumn])#]]>
            </#local.array[2][local.currentColumn]#>
          </cfloop>
        </ROW>
      </cfloop>
    </cfoutput>
  </ROWS>
</cfsavecontent>
<cfset local.xml = xmlParse(local.xmlString)>

Avoid heap error while looping in coldfusion

1 Answers