NotesStream to the test

At the NL LUG 2009 Martin Scheafer improvised during his shortened session Domino Globetrotting/Dev talk. Due to customer regulations he could not show all he had planned to show.

And so he did a couple of side steps into @function and lotusscript coding. NotesStream was mentioned as a performance booster when building (long) strings e.g. in webagents when building return data for a webservice. During execution of demo code the difference between concatening a string using [string = string + extrastring] and the same code but now using NotesStream to build the return string, concatenating was a factor 4 to 5 slower than NotesStream.
I immediately saw an opportunity to improve the performance of the several agents we have running every day to transport product and other data from Notes to our Websphere Commerce servers via xml files. Some of them deliver a xml file of over 100mb.
So I put it to the test running 4 agents delivering the same result and each of them differing slightly in how the data is written.

1. agent builds string (concatenate) to 65000+ bytes and than writes to file.
   
2. agent builds string (concatenate) per record and writes to file per record.
3. agent write to file directly, line by line.
4. agent writes to file line by line using NotesStream.

The first 3 agents write to file via LS print, agent 4 using NotesStream.

The output build has the following structure per record:
<persoon>
<voornaam>Jan</voornaam>
<achternaam>Jansen</achternaam>
<woonplaats>Scherpenzeel</woonplaats>
<persoon>

The results in performance are listed below:

1. Concatenated string test (concat string to 65000+ bytes before write).
   [Elapsed time: 296 seconds] [processed 123838 records]
2. Concatenated string test (concat/write per record).
   [Elapsed time: 163 seconds] [processed 123838 records]
3. Direct print to file test.
   [Elapsed time: 169 seconds] [processed 123838 records]
4. NotesStream string test.
   [Elapsed time: 170 seconds] [processed 123838 records]

Conclusion is, NotesStream is indeed faster, but only when building large strings. The agent (1.) that concatenates strings up to 65000 bytes proves this. When keeping the strings limited in size, agents 2, 3 & 4, NotesStream performs equal to the other string methods.
And as Martin mentioned in his session, NotesStream is certainly helpful in webagents that build return xml's on request.