How do I know when to believe the output ???
I've had the software for a little over a week now and I'm very impressed with the functionality and potential capabilities. I say "potential" because I'm having a difficult time producing confident results. I don't mean for that to come off as a cheap shot or overly negative.
I've been inputting data, modeling, re-running and finally got a product I thought represented my intent - the results looked good - and made sense at least to my inexperienced eye at this point.
I went out of town for two days - come back - rerun the software exactly as I left it - and get different results! In 2009 (for example) in two days (with no changes) my Total Spending increased $7,000 ... Total Income increased $2,500 ... Regular Assets decreased by $3000+, etc., etc. I swear - nothing was touched!!! I even kept exiting the program when I was producing the output three days ago just to insure I was working with the same, fresh data.
Is this due to the Monte Carlo modeling?
I know the amount differences aren't significant, but in my mind if I model something - like it - save it - and rerun it three days later I should get the same results.
Again - love the software and this may be a newbie learning curve issue but I'm confused.
Thanks
RSS
Yes. MC is a statistical process, depending on a stream of Gaussian random numbers (I can point you at the algorithm if you wish) for it's results. In very large MC systems, many simulations are run (100s of thousands in some cases) before producing results. The reason for the vast number of simulations is to allow all the quirks of the stream of random numbers to "work out" and produce a truly stable statistical average.
Most (I have to say most, because on-line folks who claim to provide MC results may be running their simulations on exotic iron or configurations which allow them to do 100s of thousands of simulations in something like real time) MC simulations intended for "desk top commercial" consumption run a limited number of cases. ESPlanner runs 500, which is a compromise between statistical stability and the attention span of our customers (no offense intended or implied). If MC took too long, nobody would use it or would throw out the software, if we don't run "enough" simulations the results are really suspect. In honesty, the 500 number was chosen before I came on board and I suspect may also include some technical factors, e.g., the amount of virtual/physical memory available to Windows machines and processor speed. Back in the day MC used to take 15 to 20 minutes to run and we may have to return to the 500 number at some point in the future what with 64 bit, 3+ GHz processors becoming ubiquitous.
So, yes, it IS possible to get wildly varying results from the MC without changing anything, it's the luck of the draw. It generally doesn't happen, in our experience the numbers are different, but generally close from run to run. If you really, truly haven't changed anything, this is probably what has happened. If you did change something, then it's possible there is some sort of software issue either in the UI or the CE but that's a separate discussion for the Support forum.
Best,
Dick Munroe
But if we do want to run a larger number of simulations for a longer period of time (is ESPlanner multi-core aware? E.g. will it take full advantage of a multi-core/multi-proc machine?) is there a configuration we can play with?
Unfortunately it ain't that simple. A while back, I looked into converting the MC into a threaded implementation and a substantial portion of the speed of the MC is because we can remember and communicate state between iterations, particularly during the calculation of the consumption function. It's not clear that giving up these optimizations while using multiple cores wins us anything or, more particularly, it's a boat load of work to even see if we win anything and we're a resource limited company at the moment so, given the other things on our plates, we don't have the bandwidth to perform the experiment. Personally I would love to, but ...
Even being able to vary the number of iterations has major implications for the design of the code (particularly since it's a 32 bit executable at the moment) due to the storage required for the internal state.
Best,
Dick Munroe
Thanks Dick - I'm assuming that to be the issue.
BTW - having spent 40 years in the systems engineering / programming world I too was always suspicious when a user said "I didn't change anything". But in this case - "I didn't change anything"!
Thanks again ...