Validation Running Too Long

a1newman

Validation started Friday night, and it is still running. It got past the securities price checks phase as well as other phases and is showing Validating file…

Is there a need to end task within windows? If so what do I do next? The file that is being validated is very large, but I never had it go more than 12 hours before. A simple validation was done on February 4th and that was less than an hour. That was done during the archiving of investment transactions.

Any suggestions here please.

Quicken Jasmine

Hello @a1newman,

Thank you for bringing this issue to our attention.

To diagnose this issue, it is important to determine whether Quicken is still actively processing the validation or if it has become unresponsive. Have you checked Task Manager to see if Quicken is using CPU and disk resources, or does it appear stalled? If Quicken is still consuming resources, it may still be working through the validation, though at a slower pace than expected. Additionally, how large is your Quicken file in MB, and do you have a significant number of investment accounts or historical transactions? While larger files naturally take longer to validate, it is unusual for the process to exceed 12 hours, especially if a validation completed within an hour just a few days ago.

Have you experienced any performance issues, crashes, or unexpected shutdowns in Quicken leading up to this? If so, this could indicate underlying file corruption that may be contributing to the prolonged validation. When you performed the validation on February 4th, did Quicken report any errors? It would also help to check if there were any recent Windows or security updates installed that might be impacting system performance.

If Quicken appears completely unresponsive, you may need to force-close it. Before doing so, do you have a recent backup of your data from before the validation started? If validation is interrupted, there is a small risk of data corruption, so having a backup is crucial.

Let me know!

a1newman

The file is 220 MB.

No recent Windows or security updates as last ones done wer on January 15th.

Although the task manager claims that Quicken is "not responding", the CPU percentage is changing from 12.5% to 14.1% and the Memory shows as 13.8-14.2MB. I have had the task manager claim that Quicken is not responding but then it finishes the update process anyway, so I don't take much stock in the "not responding" claim. It was identifying "not responding" earlier when it was still doing the stock prices check. That phase appears to have completed. Note that the validation done on February 4th did not include the stock prices check as it was the validation done by the system due to the archiving of investment transactions. The stock prices check phase took over 48 hours.

The report from the February 4th simpler validation with some redaction is as follows and I don't see anything strikingly wrong although others might know better:

[Tue Feb 04 22:59:46 2025]

File: "D:\Quicken\updated"

QDF:
Validating your data.
Quicken repaired Spend Condo XX[Removed-Privacy] account. No action required.
Quicken repaired Goodleap account. No action required.
Missing 3375 transaction(s) in account "XXX YYYY ####". Please check XXX YYYY #### account to verify your transaction history and reenter transactions as needed
Quicken found an invalid transaction record and removed it. "Spend Condo XX[Removed-Privacy]" 0/ 0/1900
Quicken repaired damaged transaction index. No action required.
Quicken found an invalid transaction record and removed it. "ACCT_6f" 0/ 0/1900
Quicken found an invalid transaction record and removed it. "ACCT_6f" 0/ 0/1900

Summary:
Quicken repaired 1 accounts. No action required.
Quicken repaired 3375 transactions. No action required.

QEL:
No read errors.

QEL:
All internal consistency checks passed.
Validation has completed.

Because I had some issues after the archiving including impacting a number of accounts at a bank, I am actually validating a backed-up version before the archiving was done.

I have a large number of transactions in the file as you can tell from the size of the file.

I was hoping that archiving transactions would lead to smaller file size and increased speed of updates. However, a lot of my open position are option positions (mainly short positions) with no update on prices occurring which led to inaccurate cash positions when the archiving was done. That is why I wanted to validate the prior file before using it wherein I would update the prices of my open options before doing archiving.

I am thinking to end the task using task manager if it doesn't finish by tomorrow morning.

a1newman

Hello @Quicken Jasmine

The validation finally finished. Here is what the Data_Log had (with some redactions)

[Fri Feb 07 22:21:23 2025]

File: "D:\Quicken\updated-2025-02-04.PM10.50"

QDF:
Validating your data.
Quicken repaired Spend Condo XX[Removed-Privacy] account. No action required.
Quicken repaired Goodleap account. No action required.
Quicken found an invalid transaction record and removed it. "Spend Condo XX[Removed-Privacy]" 0/ 0/1900
Quicken repaired damaged transaction index. No action required.
Quicken found an invalid transaction record and removed it. "ACCT_6e" 0/ 0/1900
Quicken found an invalid transaction record and removed it. "ACCT_6e" 0/ 0/1900

Summary:
Quicken repaired 1 accounts. No action required.

QEL:
No read errors.

QEL:
All internal consistency checks passed.

[Fri Feb 07 22:35:59 2025]
Quicken has found 3 stock split(s) for account "XX[Removed-Privacy]", security "BAKKT HOLDINGS INC COM CL A NEW", on 4/29/2024 that might be missing.
Quicken has found 2 stock split(s) for account "XX[Removed-Privacy]", security "DIREXION SHS ETF TR DAILY JR GLD MIN INDEX BULL 2X SHS NEW", on 4/23/2020 that might be missing.
Quicken has found 1 stock split(s) for account "XX[Removed-Privacy]", security "DIREXION SHS ETF TR DAILY JR GLD MIN INDEX BULL 2X SHS NEW", on 6/28/2019 that might be missing.
Quicken has found 1 stock split(s) for account "XX[Removed-Privacy]", security "DIREXION SHS ETF TR DAILY JR GLD MIN INDEX BULL 2X SHS NEW", on 5/1/2017 that might be missing.
Quicken has found 1 stock split(s) for account "XX[Removed-Privacy]", security "DIREXION SHS ETF TR DAILY JR GLD MIN INDEX BULL 2X SHS NEW", on 8/25/2016 that might be missing.
Quicken has found 1 stock split(s) for account "XX[Removed-Privacy]", security "DIREXION SHS ETF TR DAILY JR GLD MIN INDEX BULL 2X SHS NEW", on 10/1/2015 that might be missing.
Quicken has found 1 stock split(s) for account "XX[Removed-Privacy]", security "CITIGROUP INC", on 5/9/2011 that might be missing.
Quicken has found 1 stock split(s) for account "XX[Removed-Privacy]", security "APPLE INC COM USD0.00001", on 8/31/2020 that might be missing.

The following transaction(s) involving transfers appear to be damaged.
You should delete them and recreate as appropriate.

Account Standard Checking XX[Removed-Privacy]: 1/6/2025 Reverse Ach Debit 390.00

Corrected scheduled transaction: Prudential
Corrected scheduled transaction: Fidelity Investm Fidelity
Corrected scheduled transaction: Hearthwood Nb Dir
Corrected scheduled transaction: T Ins Prem
4 scheduled transactions corrected
Maximum security reference: 19436, number of securities: 19436.
Correcting invalid category type for category: ACCT_6e
Invalid category type corrected
Repair price history: 0 price(s) repaired.
[Sun Feb 09 11:33:18 2025]

Price history updated from transactions

[Mon Feb 10 17:18:22 2025]
Rebuilt investing lots.
Validation has completed.

In addition to what was in the Data_Log there was box within Quicken that had the following (with redaction):

Lot Repair Info: XX[Removed-Privacy]

Number of Lot Links – 3

Number of Links Fixed – 1

Number of Links removed - 0

I don't know what all of these messages indicate as to why the validation took so long and also whether there are further things I need to do for this file to be useful. Please advise. Thank you for your assistance.

Quicken Jasmine

Hello @a1newman,

Thank you for providing more information, I am happy to hear that the validation process finished and provided you quite a bit of information.

It looks like the validation process took an exceptionally long time due to multiple factors. Given the size of your Quicken file at 220MB and the large volume of transactions, validation can potentially be a bit time-consuming. The report indicates that 3,375 transactions were repaired, which suggests a significant amount of data processing. Additionally, the presence of investment transactions, particularly stock prices and options, likely contributed to the delay. The stock price check previously took over 48 hours, indicating issues with historical data retrieval or security price updates. The report also notes missing stock splits, which may require manual adjustments to ensure accuracy.

Several invalid transactions and damaged transaction indexes were repaired, and while Quicken states "No action required," it would be wise to review the affected accounts for missing or incorrect data. The report also mentions damaged transfer transactions, which suggests that some inter-account transfers might need to be deleted and re-entered. Another key concern is how archiving impacted your cash balances and option pricing. If options weren’t updating correctly, this could have caused inconsistencies in cash positions, making it essential to verify these values before proceeding with another archive.

Next Steps:

• Verify Critical Accounts: Check the affected accounts for any missing or incorrect transactions. Review the repaired transactions to ensure no crucial data is missing. Additionally, check any accounts with transfer transactions that appear damaged and re-enter them if necessary.
• Review Investment Accounts: Go to the investment register and verify price history, especially for open options. Manually check for stock splits and enter any that might be missing.
• Monitor Performance After Validation: If Quicken remains slow, consider creating a copy of the validated file and running another validation on the new file (File > Copy or Backup > Create a Copy). If performance does not improve, a super-validation may be necessary (Ctrl + Shift while clicking Validate & Repair).
• Decide Whether to Archive Again: Before archiving, ensure all investment prices are updated and review cash balances. Running a smaller test archive may help determine whether the issue with options persists.

If Quicken continues to experience slow performance despite these steps, let me know, and I can suggest additional troubleshooting measures!

a1newman

Hello @Quicken Jasmine

Thank you for your response. I would like to clarify a few items on my prior communications and ask questions about your response.

The report indicating that 3,375 transactions were repaired was in fact done with an earlier validation on February 4th done to archive 3,375 transactions. Given the number of archived transactions and transactions repaired were the same, it would appear that the archiving process created that result within the validation mechanism.

The stock price check took over 48 hours within this validation. In all prior validations it was less than 12 hours and probably less than 6. Why it would increase dramatically is beyond me. The only real change between this Quicken file and prior files is that now with Schwab I am getting Simple - Positions Only rather than Complete - Positions and Transactions in the Tracking Method. I chose that option because the download was otherwise providing unidentified securities requiring a lot of input for that account which was way too time consuming. Could getting simple rather complete cause such a problem of time?

Quicken repaired damaged transaction index. No action required. - Your response "Several invalid transactions and damaged transaction indexes were repaired, and while Quicken states "No action required," it would be wise to review the affected accounts for missing or incorrect data." However, it is unclear how to review the affected accounts when Quicken does not identify what damaged transaction index it repaired. Can it be assumed to be the account with the invalid transaction record that was removed right before that Quicken statement? For 2 of the 3 invalid transactions, the file reads "Quicken found an invalid transaction record and removed it. "ACCT_6e" 0/ 0/1900" However I have no such account so do not know what to do about that. Any suggestions on that?

I also wonder if I should take the Fidelity accounts and change them from Complete to Simple in the Tracking Method rather than archiving them. Can that change be done even with lots of transactions already there? Would that change help reduce time for updating and validating as needed?

Another item that isn't identified here but is new issue for me is that right after the archiving was done, a number of my accounts from one bank show in the Banking section on the left a red circle with a line through it which indicates that "Quicken in unable to update your account Click here to add your account again" As this happened right after the archiving, I went to several older backups and they all had the same result. I tried to click there and Quicken appeared to freeze on adding the account again. I tried to deactivate the online service and couldn't do so. Any suggestions on what to do with this?

Thank you very much for all your assistance.

q_lurker

The line that caught my eye was:

Maximum security reference: 19436, number of securities: 19436.

That’s more than I recall ever seeing! Does that seem like a ‘true’ value, that over the years, you have owned 19,000 different securities (each option typically being a separate security)? If so, are you seeing any slow performance behavior in investment accounts? Have you taken any measures to alleviate such issues?

a1newman

My wife is a securities trader. As such she trades daily using options almost exclusively. That is why there are so many. It would have been more except I moved to Positions only within her accounts rather than positions and transactions for the tracking method a few months ago.

q_lurker

I wonder if that ‘position only’ (Simple tracking) is a culprit? Not a lot of accumulated experience validating those files. Does that change correlate with validation issues?

Simple tracking, as I understand it, would still keep building the security list numbers, just not the transaction quantities. I suppose if she is in and out of a holding between downloads that would avoid that particular security.

FWIW: there is a long-standing suggestion from Quicken of a soft limit of 2,000 trackable securities. No one has ever clarified ‘trackable’ or the character of the softness of the limit. Others have clearly exceeded the 2000 without issue. Just not the way you have, if I recall correctly.

a1newman

@q_lurker

The huge majority maybe 98-99% of these securities are already expired. Is there any way to eliminate options that expired prior to say 12-31-2019? That old history has no value to me and eliminating such transactions would hopefully reduce the size of my file. Then on an annual basis eliminating another year would help keep the size manageable.

UKR

If you're an active trader and/or your current investment account has many transactions pertaining to sold and no longer held securities you may want to consider this:
Archive Investment Transactions
For a couple of years now, a new function has been available in Quicken, Archive Transactions.
It will split your investment account register into two parts, leaving only those securities' transactions where you currently hold >0 shares in the active register. The rest is moved to a new Archive register within the current data file.
Having less transactions in your active register might improve performance.
Please read and carefully follow instructions here: How To Archive Investment Transactions To Improve Performance
Eventually, after a couple of years, you might consider splitting your data file in two, with all the archived investment transactions moving to an archive data file and none remaining in the active data file.

q_lurker

@a1newman The archive approach presented by @UKR is the current conventional wisdom recommendation to speed up program performance for investment accounts. I tend toward a similar procedure I created before that function was introduced whereby I create a new active account and transfer only current holdings to it lot by lot.

But both those approaches were developed to address a different problem than your long slow validation. Neither of those reduce the number of securities or transactions in your file nor the size of your file.

My data file is 236 MB (per Windows, or 153 MB for the QDF file per Quicken Help) compared to your 220 MB. My file goes back 30+ years and has 1500+ securities. I’ve had no validation timing issues. Never more than 10 minutes, as best I recall.

The QDF file you see in Windows is actually a compressed Zip file comprising 5 or so working files. You can see that compressed file structure with the 7-zip program. If you want to dig that direction, I’d be curious what it shows for the internal QPH file. Mine shows almost 72 MB compressed = about 1/3 of the overall file. The QPH file is where the security price history data is written. The internal QDF file is the 156 MB Quicken shows. Because your security count is so high and where your validation hangs, I wonder about the size of your QPH file and wonder if that is somehow out of whack or an issue.

a1newman

@q_lurker

The QPH is 12MB and the QDF is 168MB. I suspect that the QPH is not that large because much of the downloads on options come is as unidentified securities and I entered the price when the option was opened and then when it was closed. Thus, not a lot of history on those securities. If they came in as identified options, many of those would close the same day or the next day producing little price history.

The archive investment transactions process doesn't identify whether it would work on an account where I started it as complete tracking (positions and transactions) and then changed to positions only. Is there any way to confirm that is not going to be a problem, Further I would think I would need to add current prices on the open options, or I suspect I would get odd results.

a1newman

@q_lurker

Given the problems I have with the time Quicken takes to do things and broken linkages, I have decided to restart with a new file and do my transactions simple rather than complete on my more complex accounts. I really don't use Quicken transactions for taxes and rely upon the investment firm to provide what I need. I will see once that is done whether I continue to have difficulties. If I do, that will indicate Quicken is apparently broken. Thanks for your assistance.

q_lurker

That certainly seems reasonable.

A question not asked seemed to be, why were you running the validate to begin with. I reread your first post and you mentioned once as a part of archiving investments. That is forced probably for good reason.

Prior to that, were you validating with the same lengthy problems? Apparently not, though I’d also consider an hour borderline problematic.

What problems were you having that led to the archiving? Any point to restoring pre-archive and going from there?

Finally any thing in particular change between Feb 4 archiving and Feb 7 3-day validation? Is that when you went to simple tracking?

a1newman

@q_lurker

When I did the archiving, I didn't have the prices of the options that had come in as unidentified in the transactions and therefore needed me to identify them updated to current pricing. The end result was that the IRA account after archiving had negative cash which is not possible.

I also found that the PNC accounts I had were not properly updating and I couldn't get that corrected. That problem with PNC also happened when I brought backups back and I tried several backups with all having the same problem.

The validating lengthy time problems all appeared to relate to securities price checking. In October it took maybe 10 hours. This time was crazy long. When the validation was done for the archiving process, it didn't check the prices, and it took a reasonable time (maybe 10 minutes).

Nothing had changed during the period other than including the securities price checking.

q_lurker

I may have posted that last message too quickly, because my real message and my policy is to not do the validate and repair unless there is a real need to. Further to that, I have never done the ‘Correct investing price history’ because:

• It has never been identified how or against what criteria Quicken would identify a problem.
• I cannot risk having Quicken altering prices of securities that lack valid tickers or are otherwise irreplaceable.
• Quicken is terrible about being specific about changes. I would not trust it to tell me they changed the price of this security on this date from that value to this value. Without that level of specificity, I could not begin to assess the correctness of their repair.

I suggest if you can live without making that selection on a validation, you may be able to continue with your current file rather than start a new one.

a1newman

@q_lurker

I appreciate your comments about 'Correct investing price history' However I was doing the validation due to the overall slowness of Quicken with the hope of resolving that.

I have started the new file and have some work to do on it before it is finished but it is much faster and so far, no issues.