Can I force Quicken to re-import a month's data from scratch?

wilsonch

After a series of mishaps with web connect, my primary checking account transaction data is in bad shape. For instance, my 2020-10 data was missing many transactions but also had a bunch of duplicates (created by an officially recognized Quicken import/web-connect bug). I got a fresh QFX file for the month and imported it. It imported some data but ignored a bunch of transactions. I verified by looking inside the QFX file that the transactions were there, but Quicken wouldn't import them. I entered them manually.

I thought for 2020-11 that as a test, working in a throw-away copy of my Quicken file, I'd delete the month's transactions and import a fresh QFX file of the month's data. (The deleted transactions had "C" reconciliation status, not "R" status.) But even though there are no 2020-11 transactions at all in the account's register Quicken declines to import any of the transactions contained in the 2020-11 QFX file. There is a delay of a second or so and nothing happens. No error messages.

Is there a way I can convince Quicken to import these transactions?

wilsonch

I'm going to assume that this in an accurate description of Quicken's designed / intended behavior - that Quicken is to remember the FITIDs it has imported and silently refuse to re-import them.

My working conclusion from my experience and from this statement of Quicken's intended behavior is that I should abandon my current Quicken file and load the account history data into a completely empty new file. *The rest of my message will explain my reasoning and may or may not be interesting or useful.*

Background: My current data file was populated by Quicken Web Connect directly from the financial institution. The data was processed by Web Connect updates over a period of several weeks. At least some of these runs were by buggy code (see #2 below). The Quicken data file validation utility finds no problems in the data file. In the months I examined I found four sets of transactions.

1. Accurately imported transactions (approximately 70% of transactions)

2. Duplicated transactions (approximately 10% of transactions). My understanding from a message sent by Quicken is that these duplicates were introduced by bugs in the Web Connect processing. The bugs supposedly operated a week or two and were fixed. The Quicken message reporting the bug fix said users would have to delete the duplicates manually, identifying them visually. (The two copies were not pure duplicates -- some of the text fields were different.)

3. Rows not existing after Web Connect processing but which were imported okay from a QFX file manually downloaded and imported. (maybe 10-15% of transactions)

4. Rows not existing after Web Connect processing but which Quicken refused to import from a QFX file manually downloaded and imported. (maybe 5-10% of transactions) These are the rows that supposedly are being excluded based on an FITID match.

For the FITID rule to explain group 4 we would have to construct a narrative in which Web Connect processing deleted some rows it had already imported. I assure you I did not randomly delete 10% of transactions. I don't even know how one would do such a thing. A more likely explanation in my view is that there are or were bugs in the FITID-checking code or that the internal FITID list is corrupt so that Quicken incorrectly concludes that a certain FITID has already been imported.

Alternative courses of action:

1. Manually edit large XML files to change hundreds of values of a supposedly unique identifier that obeys unknown validation rules, then import the data and manually verify that no transactions were omitted.

2. Manually reconcile the Quicken data with the bank statements, entering / fixing data as required.

3. Abandon the current Quicken file (which contains long history and multiple accounts) and use Web Connect to load selected accounts into a brand new file from scratch.

4. Coax Quicken support into providing a way to turn off FITID validation. As a retired programmer, I'm 90% sure there exists such a way.

5. Completely abandon using Quicken. Move to some other software.

Probably I will embark on option #3.

splasher

Quicken remembers the FITID (unique identifier) of each transaction on an account basis, so that it will not import duplicate transactions.

If you want to override that behavior with a Web Connect .QFX file import, edit the .QFX file and add a letter or number to the FITID field in each transaction so that Quicken will see it as a "new" transaction.

wilsonch

I'm going to assume that this in an accurate description of Quicken's designed / intended behavior - that Quicken is to remember the FITIDs it has imported and silently refuse to re-import them.

My working conclusion from my experience and from this statement of Quicken's intended behavior is that I should abandon my current Quicken file and load the account history data into a completely empty new file. *The rest of my message will explain my reasoning and may or may not be interesting or useful.*

Background: My current data file was populated by Quicken Web Connect directly from the financial institution. The data was processed by Web Connect updates over a period of several weeks. At least some of these runs were by buggy code (see #2 below). The Quicken data file validation utility finds no problems in the data file. In the months I examined I found four sets of transactions.

1. Accurately imported transactions (approximately 70% of transactions)

2. Duplicated transactions (approximately 10% of transactions). My understanding from a message sent by Quicken is that these duplicates were introduced by bugs in the Web Connect processing. The bugs supposedly operated a week or two and were fixed. The Quicken message reporting the bug fix said users would have to delete the duplicates manually, identifying them visually. (The two copies were not pure duplicates -- some of the text fields were different.)

3. Rows not existing after Web Connect processing but which were imported okay from a QFX file manually downloaded and imported. (maybe 10-15% of transactions)

4. Rows not existing after Web Connect processing but which Quicken refused to import from a QFX file manually downloaded and imported. (maybe 5-10% of transactions) These are the rows that supposedly are being excluded based on an FITID match.

For the FITID rule to explain group 4 we would have to construct a narrative in which Web Connect processing deleted some rows it had already imported. I assure you I did not randomly delete 10% of transactions. I don't even know how one would do such a thing. A more likely explanation in my view is that there are or were bugs in the FITID-checking code or that the internal FITID list is corrupt so that Quicken incorrectly concludes that a certain FITID has already been imported.

Alternative courses of action:

1. Manually edit large XML files to change hundreds of values of a supposedly unique identifier that obeys unknown validation rules, then import the data and manually verify that no transactions were omitted.

2. Manually reconcile the Quicken data with the bank statements, entering / fixing data as required.

3. Abandon the current Quicken file (which contains long history and multiple accounts) and use Web Connect to load selected accounts into a brand new file from scratch.

4. Coax Quicken support into providing a way to turn off FITID validation. As a retired programmer, I'm 90% sure there exists such a way.

5. Completely abandon using Quicken. Move to some other software.

Probably I will embark on option #3.

Rocket J Squirrel

6. Create a new empty QDF file. Import the QFX into it. Delete any unneeded transactions, retaining only the ones you need to add to your original file. Export to QIF from the new file and import the QIF into the original file.

Chris_QPW

EDIT Changed order of deleting account.

7. Deactivate the account for downloading.  Import QXF file into a new account.  Move transactions needed to the old account.  Delete new account.  Reactivate old account.

wilsonch

I thank everyone for wading through my long post and offering thoughtful suggestions!

Even though my post was very long it sorta left some info out, namely that I'm not just cleaning up one month but (say) a year's history for my two most active accounts (checking and PayPal, which are linked by many transfers), so I can make a budget and project expenses forward. I settled upon the idea of cleaning up the data a month at a time. I manually fixed 2020-09 and 2020-10. I decided not to tackle 2020-11 the same way.

Manual reconciliation and data correction took a long time and I'm going to avoid it if I can. Hence possibilities such as 1, 2, and 6 are ones I want to avoid. ("Delete any unneeded transactions" is the kind of thing I'm trying to avoid -- selective manual intervention.) What I want is a clean, simple, total load of data for time period X of account Y. Either Web Connect loads the data or I do, I verify that the grand totals for the beginnings and ends of the statement time periods match up, and I'm done. No transaction-by-transaction crawl through the bank statement and the data, which is what I did for the two months I cleaned up.

It may be that I'm being unreasonable to decide to move to a brand new file, but right now I think to continue in the same file is to continue to work with whatever set of conditions confused Web Connect such that it munged my data. Even though the validation utility doesn't find any hard errors in the file, there may be soft errors (such as perhaps FITID table / algorithm errors). The earliest transaction date in the file (from a checking account other than the one I've been talking about here) is 1994-08-21. So I've been using Quicken as long as splasher.

Thanks again, everyone.

gdhoffman7013

I recently had the somewhat common experience of needing to re-import data into Quicken because my Credit Card bank produced a QFX file with the incorrect dates. Of course Quicken would not let me re-import the data from that time period because of the for-mentioned issues (I presume). What did work was to override the existing file with the restored recent backup file. I could then generate a correct QFX file from my Credit Card back and import it into Quicken. Fortunately, other than import another QFX file, which was easy to redo a second time, I had made no changes to the working file from when I created the backup, .