update a file in node-red through context-storage

Question

I have written code to update a file in node-red that works as I want it to. I am not too sure if it is realiable though. When pressing the inject button the current value is stored through context, a file is read and the stored data added to it. What about (a-)synchronicity? What if the stored data becomes larger? maybe the data will not have been stored yet, but the older data is written to the file? is that a possibilty? could i solve this by using some kind of callback function somehow?

here is the node-red code:

[{"id":"ff428878.2bf0d8","type":"inject","z":"33a0c3f0.5ad66c","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"x":140,"y":80,"wires":[["d97f4731.48f508"]]},{"id":"aff66bbc.653048","type":"file in","z":"33a0c3f0.5ad66c","name":"","filename":"examplefile.txt","format":"utf8","chunk":false,"sendError":false,"x":380,"y":240,"wires":[["82b889ff.4e5f18"]]},{"id":"216dc166.5c79be","type":"file","z":"33a0c3f0.5ad66c","name":"","filename":"examplefile.txt","appendNewline":true,"createDir":false,"overwriteFile":"true","x":900,"y":80,"wires":[]},{"id":"9238c649.dac2e8","type":"function","z":"33a0c3f0.5ad66c","name":"merge","func":"if(msg.topic == \"new\"){\n context.set('shortterm', msg.payload);\n return [null,msg];\n}\n\nif(msg.topic == \"old\"){\n msg.payload.push(context.get('shortterm'));\n return [msg,null];\n}\n\n","outputs":"2","noerr":0,"x":611,"y":80,"wires":[["216dc166.5c79be"],["aff66bbc.653048"]]},{"id":"d97f4731.48f508","type":"function","z":"33a0c3f0.5ad66c","name":"mark as new","func":"msg.topic = \"new\";\nreturn msg;","outputs":1,"noerr":0,"x":400,"y":80,"wires":[["9238c649.dac2e8"]]},{"id":"10b8938f.1cddfc","type":"function","z":"33a0c3f0.5ad66c","name":"mark as old ","func":"msg.topic = \"old\";\nreturn msg;","outputs":1,"noerr":0,"x":810,"y":240,"wires":[["9238c649.dac2e8"]]},{"id":"82b889ff.4e5f18","type":"function","z":"33a0c3f0.5ad66c","name":"JSON parse","func":"if(msg.payload){\n msg.payload = JSON.parse(msg.payload);\n}\n\nreturn msg;","outputs":1,"noerr":0,"x":610,"y":240,"wires":[["10b8938f.1cddfc"]]},{"id":"7572637c.0a25ac","type":"file","z":"33a0c3f0.5ad66c","name":"","filename":"examplefile.txt","appendNewline":true,"createDir":false,"overwriteFile":"true","x":510,"y":480,"wires":[]},{"id":"1f6b9043.ff153","type":"inject","z":"33a0c3f0.5ad66c","name":"","topic":"","payload":"[]","payloadType":"json","repeat":"","crontab":"25 22 * * *","once":false,"x":210,"y":460,"wires":[["7572637c.0a25ac"]]}]

Answer 1

If you are using a single flow to read the file, the next step in the flow will not happen until the file is read.

So your flow is too complex.

inject->read file->json node (if the file content is json)->
  function node->write file

where the function node:

1. gets the context variable (or creates an empty one if it doesn't exist)
2. Merges the file data with the context
3. writes the context variable
4. changes the msg.payload to the updated content
5. return msg

Answer 2

Another newer option for persistent file storage with Node-Red 19.0+ is the built in "context storage".

There is a nice write up on how to get rolling with context storage here:

https://discourse.nodered.org/t/a-guide-to-understanding-persistent-context/4115

The basics are to update your ~/.node-red/settings.js file with contextStorage settings:

contextStorage: { default : { module: "memory" }, fileStore: { module: "localfilesystem"} },

From there you can reference in your flows:

flow.set("jsObj", "Hello there!", "fileStore");

To get data back out:

flow.get("jsObj", "fileStore")

At which point you can see your saved file within your ~/.node-red/context/ folder.

The linked article above should be able to answer any other questions.