Quicken is slow to delete investment transactions
Watching the performance monitor while it's deleting, I see the CPU is barely used (about 6-7%), plenty of memory available, and the Disk I/O barely registers (about 50KB/second). I suspect the program's interaction with data is the culprit (e.g. excessive table indexing) but have no way to confirm this. The Quicken file is minuscule, so I'm baffled why the program can't use all the PC's resources. What's the constraint?
One peculiar thing I see happening is that the program is refreshing the screen after each and every delete. The last thing it does is refresh the report, again one line at a time. Both these functions seem unnecessary and inefficient.
The problem of slow response time, as most long-time users are painfully aware, is nothing new. I've tried a variety of techniques to reduce the number of transactions (including completely new accounts) but after a few years, it gets slower.. and slower.... and slower...... and slower.
I've validated the file and tried all the common techniques. I'm posting this question hoping someone from Quicken with a technical background can comment. Cheers.
Comments
-
I seriously doubt any person from Quicken is going to comment on this.
This is in the realm of the developers, and they don't come here.
As a 35+ years programmer I will give you my take on it.
On the report and doing things like delete, changing categories, ...
Basically the programmer took the easy way out and wasn't concerned about efficiency.
In other words, they found a function to do "one" and simply put that in a loop for each transaction selected. Instead of coming up with a way to just issue a delete command for all the transactions at once and then when they come out of that have the report refresh one time.
But I might point out that the reason people use the banking report(s) to change investment transactions is because they never bothered to put in these functions into the investment registers.
This "GUI" approach is seen all through Quicken. As in lots of things are implemented as if a user is doing something one click at a time, instead of calling something at a lower level that can do the work and return.
On the slowness of the investment account vs a banking account.
You really can't compare the two. They have very different jobs to do.
If you look at the banking register I can take any of the transactions based on any filters, put them in the GUI and the only thing that needs to be computed is the running balance.
If you look at the investment register you have multiple securities, with multiple lots that all have to be connected/computed for each transaction shown. There is just a lot more data, and computing to do.
But on top of that I also believe that the GUI components, in the investment register, is constructed in an inefficient manner.
Could this all be made to be faster by precomputing information, and using a better GUI layout, and overall better data structures?
I certainly think so, but it actually isn't the point.
People tend to think that work is optimize to get to the "best".
But instead it works like evolution, it is optimized to "just good enough".
Everything takes time to do, and as such they have to decide what is important at that moment in time. If the investment registers are "OK" for 95% of your users you don't go in and optimize to make it work better for the 5%. You have much more pressing matters elsewhere.0 -
P.S. "Watching the performance monitor while it's deleting, I see the CPU is
barely used (about 6-7%), plenty of memory available, and the Disk I/O
barely registers (about 50KB/second). I suspect the program's
interaction with data is the culprit (e.g. excessive table indexing) but
have no way to confirm this. The Quicken file is minuscule, so I'm
baffled why the program can't use all the PC's resources. What's the
constraint? "
It is baffling. And there is basically only three possible answers.- Programmed waits.
- Waiting on hardware.
- Using resources that don't show up in the task manager.
#2 Waiting on hardware (like the GPU) isn't recorded in any process.
#3 This is basically #2 but also recognizing that there some parts of the operating system that are so low level they don't track them in the Task Manager because they aren't associated with any process.0 -
How do you make that File Information display shown above? Does your system only have 2 gigs of memory, or am I reading it wrong?QWin Premier subscription0
-
OK, I figured that its is Help then Ctrl-click on About Quicken. But the memory reported is weird - My system has 8 gigs but the amount available shows as 2097151, exactly the same as Fred's report above.Jim Harman said:How do you make that File Information display shown above? Does your system only have 2 gigs of memory, or am I reading it wrong?
QWin Premier subscription0 -
Quicken is a 32-bit program as such the typical max memory it can allocate is 2GB.Jim Harman said:How do you make that File Information display shown above? Does your system only have 2 gigs of memory, or am I reading it wrong?
There is some exceptions to the rule for the right kind of hardware Quicken does seem to be able to get up to about 3.5GB (which is the absolute for a 32-bit Windows program).
But all of that is pointless. Quicken consumes less memory than your web browser.
The only time I have ever seen Quicken consuming anything close to 1GB of memory is when there was a memory leak in Quicken 2015's QIF import, and it was on its way to crashing because it ran out of memory.0 -
Got it. Thanks for the clarification.Jim Harman said:How do you make that File Information display shown above? Does your system only have 2 gigs of memory, or am I reading it wrong?
I agree some operations appear to be simulating key clicks in the GUI. For example moving shares between accounts when there are a lot of tax lots involved is very slow.
In most cases these are things you (hopefully) don't do very often, so I guess the ability to do it at all is worth the wait.QWin Premier subscription0 -
BTW at one point I needed to delete thousands of investment transactions out of a few accounts, and that was taking hours, so I worked out a way to speed it up quite a bit.
What I did is select a few hundred transactions from the end of the report, and then repeat.
The reason this speeds it up is because say you select all them. What they are going to do is delete the first transaction, and then the report is going to be refreshed. Well this means that they have to recompute all the numbers from the 2 transaction onward. If you delete the last transaction in the report it doesn't have to recompute anything after that point. So limiting how far back you delete limits how much it recomputes for each refresh.0 -
Very true. Given the prevalence of cloud computing, the Windows product may be near the end of its platform life cycle anyway.QPW said:I seriously doubt any person from Quicken is going to comment on this.
This is in the realm of the developers, and they don't come here.
As a 35+ years programmer I will give you my take on it.
On the report and doing things like delete, changing categories, ...
Basically the programmer took the easy way out and wasn't concerned about efficiency.
In other words, they found a function to do "one" and simply put that in a loop for each transaction selected. Instead of coming up with a way to just issue a delete command for all the transactions at once and then when they come out of that have the report refresh one time.
But I might point out that the reason people use the banking report(s) to change investment transactions is because they never bothered to put in these functions into the investment registers.
This "GUI" approach is seen all through Quicken. As in lots of things are implemented as if a user is doing something one click at a time, instead of calling something at a lower level that can do the work and return.
On the slowness of the investment account vs a banking account.
You really can't compare the two. They have very different jobs to do.
If you look at the banking register I can take any of the transactions based on any filters, put them in the GUI and the only thing that needs to be computed is the running balance.
If you look at the investment register you have multiple securities, with multiple lots that all have to be connected/computed for each transaction shown. There is just a lot more data, and computing to do.
But on top of that I also believe that the GUI components, in the investment register, is constructed in an inefficient manner.
Could this all be made to be faster by precomputing information, and using a better GUI layout, and overall better data structures?
I certainly think so, but it actually isn't the point.
People tend to think that work is optimize to get to the "best".
But instead it works like evolution, it is optimized to "just good enough".
Everything takes time to do, and as such they have to decide what is important at that moment in time. If the investment registers are "OK" for 95% of your users you don't go in and optimize to make it work better for the 5%. You have much more pressing matters elsewhere.0 -
That makes sense, but I would have expected the CPU would be screaming during all the re-computation. However, as you pointed out, if it's just simulating a "one-line" click over and over, the CPU or disk I/O may not be the constraint.QPW said:BTW at one point I needed to delete thousands of investment transactions out of a few accounts, and that was taking hours, so I worked out a way to speed it up quite a bit.
What I did is select a few hundred transactions from the end of the report, and then repeat.
The reason this speeds it up is because say you select all them. What they are going to do is delete the first transaction, and then the report is going to be refreshed. Well this means that they have to recompute all the numbers from the 2 transaction onward. If you delete the last transaction in the report it doesn't have to recompute anything after that point. So limiting how far back you delete limits how much it recomputes for each refresh.
With enough PC power to read the entire file in about 0.1 second, it's frustratingly slow. The networked version must be completely different, as it could never get away with that kind of "chatty" transaction.
0 -
Personal I have never "understood" GUIs requirements for being so inefficient.QPW said:BTW at one point I needed to delete thousands of investment transactions out of a few accounts, and that was taking hours, so I worked out a way to speed it up quite a bit.
What I did is select a few hundred transactions from the end of the report, and then repeat.
The reason this speeds it up is because say you select all them. What they are going to do is delete the first transaction, and then the report is going to be refreshed. Well this means that they have to recompute all the numbers from the 2 transaction onward. If you delete the last transaction in the report it doesn't have to recompute anything after that point. So limiting how far back you delete limits how much it recomputes for each refresh.
My career was more "systems" as in the core code. I have seen where fetching 100,000 record from a database, even more lines of data from a data file can be done in basically the blink of an eye.
But with most GUIs if you put more than say 1000 items in a list without it being "virtual", it will ground to a halt!
I can give you an example of how you use very little CPU time and still be waiting.
Say I have a large spreadsheet and change one number on it and then call the system to refresh the graphics. Now I have is situation where the time the CPU took to do the number change could be much shorter than the time the refreshing of the graphics takes. Now do that a million times.
What you will see is very little CPU time, but you can still be waiting around. The Task Manager doesn't show non disk/network I/O.0 -
I never did find a reason why transaction deletes are so
slow, but I found two work around solutions.
Both solutions create a new, smaller account that is faster to work with
from that point forward. The basic strategy is to:A.
Create a new account which will be used to
receive open investments from an existing account.B.
Transfer the investment information from the old
account to the new account. This can be done either in summary or detail (see below).C.
Activate online services for the new account,
Deactivate the old account.Obviously, back-up your data before starting.
Transfer In Detail: This method automatically transfers
investments from one account to another using a built in Quicken
function. NOTE: There is very little
consolidation of information with this function. All dividend and buy/sell
history for any security with a non-zero share balance is transferred to the new account.
1.
From the Investment register, click Enter
Transactions -> Shares Transferred Between Accounts. Select either One
Security or All Securities.2.
Enter the new account which will receive the
securities.Transfer In Summary: This method manually transfers investments, in summary, from one account to another to produce a new account with minimal information about each security (just share
balance and cost basis). No lot information or history is moved. The history is still available for reporting and such, so long as you don’t delete the old account.1.
Run a Quicken report to produce a summary list of
investments, including the shares and cost basis of each security. Reports -> Investments ->
Portfolio Value2.
Using the report, add the total shares of each
security to the new Account. Enter
Transactions -> Add – Shares Added3.
Using the report, remove the shares from the old
account. Enter Transactions -> Remove
– Shares Removed4.
Run the Portfolio Value report on the new
account to confirm the totals match and all the securities were moved
correctly.0
