Need a tool to really, truly, repair data corruption (13 Merged Votes)

Options
13»

Comments

  • Scooterlam
    Scooterlam SuperUser, Windows Beta Beta
    edited July 2021
    Options
    <deleted>  This message was in response to a now moderated / merged post.

  • KlisterKid
    KlisterKid Member ✭✭✭
    Options
    Scooterlam .... not only does the validate fail to fix the categories, but it also "trashes" the categor(ies); e.g., I'll see a description for the category that has text from a random entry when I have purchased a stock, a grocery item, whatever. I've done repeated validates to see if my "fixes" to the categories' descriptions are permanent; NO WAY. At least I have not had a repeat of the tags merging into other sub-categories ... a real nightmare to fix. WHAT I HAVE DONE (and it doesn't seem to be helping that much) is to find identical tags and added a unique suffix or prefix to that category; e.g., instead of just FUEL, I added FUEL-AUTO to distinguish it from fuel for a gas grille, etc. even though the super-category is unique. I'll let folks know if what I've done is anything worthwhile.
  • rgumpertz
    rgumpertz Member ✭✭✭✭
    Options
    That is exactly why a reliable Validate & Repair tool is needed to "really, truly, repair data corruption"!
  • studpup
    studpup Windows Beta Beta
    Options
       Fixed damaged category.  Please check your category list by going to Tools>Category List.
       ...
       Super validation has completed.

    I did, everything looks fine, immediately run validate again, same error again. Not sure the problem was "fixed".

    wags Glad to be here!

  • jpayneiv
    jpayneiv Member ✭✭
    Options
    I have not run the validation tool in a long time. I saw some oddities in one of my accounts so I gave it a try. I backup my data then ran validate.

    OMG I go this:

    QEL:
    The old file was corrupt and only some of the data has been recovered.

    In an attempt to fix the problems the validation tool rendered the file useless! After a panic attack, I am looking for a solution to get some direction on recovering my 15 years of data! I have EVERYTHING in quicken and rely on it.

    Are there any suggestion on how to pick up the pieces?
  • q_lurker
    q_lurker SuperUser ✭✭✭✭✭
    Options
    @jpayneiv. The validation routine should have created a ‘before’ backup of the file. Check your file folder with the file for a Validation sub-folder. 
  • Ps56k2
    Ps56k2 SuperUser ✭✭✭✭✭
    Options
    hmmm - is this a Voting issue or not ?
  • Rocket J Squirrel
    Rocket J Squirrel SuperUser ✭✭✭✭✭
    edited March 2022
    Options

    Ps56k2 said:
    hmmm - is this a Voting issue or not ?
    Why yes, it is.

    Well, it was until being marked "planned."
    httpsexternal-contentduckduckgocomiuuhttps3A2F2Fipinimgcom2Foriginals2Fb72Fa72F152Fb7a71512232f152e7325662f823effbajpgf1nofb1

    Quicken user since version 2 for DOS, now using QWin Biz & Personal Subscription (US) on Win10 Pro.

  • Chris_QPW
    Chris_QPW Member ✭✭✭✭
    Options
    When I read this request, I have to wonder where all the SuperUsers have been for the last 30 years.

    Such trust in the Quicken developers with such an open-ended statement.

    To me it is almost like asking the politicians "to fix the system".
    Signature:
    This is my website: http://www.quicknperlwiz.com/
  • studpup
    studpup Windows Beta Beta
    Options
    Chris_QPW said:
    When I read this request, I have to wonder where all the SuperUsers have been for the last 30 years.

    Such trust in the Quicken developers with such an open-ended statement.

    To me it is almost like asking the politicians "to fix the system".
    It's not a single request, to me it's a new approach: Right now any problem that can't be reproduced in a blank file becomes 'just restore your data file'. But a new approach would be to submit the details for someone to inspect that user's data file, figure out how to detect/correct that specific kind of problem, and improve the Validate routine in the next release. 

    But for wish list, I'd love to see the Validate routine log all the data it changes instead of just saying "corrected". I'd like more detailed log file specifically showing anything it changes, and a warning if it changes the amount any reconciled transactions/accounts. Please don't change my reconciled balances "silently"

    wags Glad to be here!

  • Chris_QPW
    Chris_QPW Member ✭✭✭✭
    Options
    Studpup said:
    Chris_QPW said:
    When I read this request, I have to wonder where all the SuperUsers have been for the last 30 years.

    Such trust in the Quicken developers with such an open-ended statement.

    To me it is almost like asking the politicians "to fix the system".
    It's not a single request, to me it's a new approach: Right now any problem that can't be reproduced in a blank file becomes 'just restore your data file'. But a new approach would be to submit the details for someone to inspect that user's data file, figure out how to detect/correct that specific kind of problem, and improve the Validate routine in the next release. 

    But for wish list, I'd love to see the Validate routine log all the data it changes instead of just saying "corrected". I'd like more detailed log file specifically showing anything it changes, and a warning if it changes the amount any reconciled transactions/accounts. Please don't change my reconciled balances "silently"
    And what makes you believe that the Quicken developers are thinking the same way you are?

    I could list tons of requests where the people submitting the request thought they were going to get one thing and the Quicken developers decided to do it in a completely different way.

    Without exact specifications, that is about all one should expect.

    Also even though I definitely agree that the reporting on Validate and Repair could be a lot better, and even there is a possibility of some user interaction to help the process, anyone that believes that Validate and Repair improvements are the solution to this problem is naive or doesn't really understand what is going on.

    When data is corrupted, there may be no solution other than deleting that data.  And even determining that isn't straightforward.

    One thing that the user never sees is that complicated relationships that are in the database and such.

    Validate and Repair is based on checking links and such, and "illegal values", and also from "experience".
    As in some kind of problem is reported, and they decide they can look for that given problem and apply a given set of steps to try to fix it.  This isn't "proactive" it is reactive.  So, for your "asking the user" kind of problem they have already know that their is a possibility of such a problem, and that they don't have enough information to do it automatically and then ask.  That is a one off kind of "solution", not anything that you can generalize.

    The truth is that there is only one true "solution" and there isn't any way Quicken Inc is going to do it.

    That is to create an export and import facility that is in a human readable format, because humans are the only ones that could look at given set of data and decide the best action for cleaning it up.  Maybe if Quicken Inc tons of money are expertise it might be possible to setup this in AI, but that certainly off the table.

    So, why do I believe that they will never do it?

    1. It is LOT of work to do this.  This is a major feature, not a minor one.
    2. It opens the door to competitors leveraging this system for their benefit.
    3. The only reason it is better than just adding to QXF exports/imports, is because it gives the people with a lot of skill will be able to use it to fix their problems.  But this will leave out the majority of Quicken users.
    4. Note that a "full" QXF export/import would be an "actionable" step, that might be more plateable to Quicken Inc.  But again, this would "help", but no be a "tool to really fix corruption".  This already exists in Quicken Mac, and helps them a lot.  But it works only because as you convert from one format to another the reading, validating, converting, writing just naturally removes "some kinds" problems, but certainly not all.  In other words one can expect a good new file, but not necessarily functional equivalent file (data is missing from it). 
    Signature:
    This is my website: http://www.quicknperlwiz.com/
  • Rocket J Squirrel
    Rocket J Squirrel SuperUser ✭✭✭✭✭
    Options
    Chris_QPW said:

    The truth is that there is only one true "solution" and there isn't any way Quicken Inc is going to do it.

    That is to create an export and import facility that is in a human readable format, because humans are the only ones that could look at given set of data and decide the best action for cleaning it up. 
    • It opens the door to competitors leveraging this system for their benefit.
    This is the sum of it. I requested this years ago and it was denied.

    https://community.quicken.com/discussion/7776408/add-export-import-to-from-text-files-for-purpose-of-identifying-and-removing-corruption

    Quicken user since version 2 for DOS, now using QWin Biz & Personal Subscription (US) on Win10 Pro.

  • genoajoe146
    genoajoe146 Member ✭✭
    Options
    Functionality seems to be degrading continually....the infuriating data corruption, failure to display data have me looking for an alternative platform.
  • whodiini
    whodiini Member, Windows Beta Beta
    Options
    Since there is a profit motive behind Quicken's refusal to create an export/import capability, then perhaps Quicken could sell that app. I would be willing to pay $100 for such an app. Like many people, I know my file has corruption that Quicken validate and supervalidate does not catch.  If there is a smaller group of people who need it, then maybe for $100 per user it might make it worth Quicken to do. 
  • Philip107
    Philip107 Member ✭✭✭✭
    Options
    I think a routine to create a new data file based on the existing accounts and other settings might work out okay to start over and leave the corruption behind. Other sources behind bad data are not corruption but just outright bugs in the software that happen under certain conditions.

    ----Quicken User since 1998 ----

  • FlyerFran
    FlyerFran Windows Beta Beta
    Options
    Data Integrity trumps everything else in level of importance in Quicken.  All the features are meaningless if my data file needs to be rebuilt.  
  • Rocket J Squirrel
    Rocket J Squirrel SuperUser ✭✭✭✭✭
    edited October 2022
    Options
    This suggestion from March 2019 has been marked "Planned". I would like to see an update, especially since being "Planned" prevents users from further voting it up.

    Quicken user since version 2 for DOS, now using QWin Biz & Personal Subscription (US) on Win10 Pro.

  • UKR
    UKR SuperUser ✭✭✭✭✭
    Options
    This suggestion from March 2019 has been marked "Planned". I would like to see an update, especially since being "Planned" prevents users from further voting it up.
    I concur. This needs to be completed, sooner rather than later.
    Since the 1980s I keep hearing about a California High Speed Rail System from L.A. to San Francisco. We're all still waiting for it. By the time they finish it and we can take a train ride on it, we're all going to be well past our 100th birthday.
    Let's make sure that this process doesn't take as long, shall we?

  • Pumphouse
    Pumphouse Member ✭✭✭
    Options
    I fully agree with the intent of this request. But it may be a heavier lift for the Quicken team than we realize. (The following are my speculations based on 35 years of software engineering in database applications. I could be very wrong. Also, none of this is intended as criticism. I've been there.)

    Quicken started on DOS and was ported to Windows. I suspect it had a proprietary database engine and the Windows port kept it. But it's old, and probably quirky. And more to the point, the original guys that understood it are probably long gone. The new team probably (and rightly) are afraid to change much - who knows what will break?

    When they ported to the MAC, SQL database engines were cheap and good. I think they made the right choice to use SQLlite on the MAC even though not all features ported right away. So much development in integrity and tools for free.

    So maybe we should be hoping they fix the data integrity issues by backporting the Mac code to Windows.
  • Ken McAdams
    Ken McAdams Member ✭✭✭✭
    Options
    Pumphouse - well said. Thanks for your input.
  • Robert Chiarulli
    Robert Chiarulli Member ✭✭✭
    Options

    I run validate and repair regularly to clean up any errors early. Occassionly online payments I make from my bank to a credit card gets removed from the bank ledger and the transaction in the credit card ledger the category is changed to the credit card account.

  • Q_M
    Q_M Member
    Options

    Data integrity problems persist. I have had to rebuild my Quicken data file twice in the last three months. I always have to go back to a backup that is at least six weeks old (backup daily—manually and automatically). Quicken Team, FIX THIS DATA INTEGRITY PROBLEM NOW!!! I am beyond tired and frustrated with this issue. I have talked to the Quicken Team and the answer has been consistent "Reset the data file and restore from a know good point and rebuild the lost transactions".

  • Jay Gourley
    Jay Gourley Member ✭✭✭
    Options

    I think this thread is about the Validate routine. I believe it does repair some problems. It does not rebuild a new problem-free data file as the originator of this thread suggested back in 2019. I just spent 2.5 days rebuilding a data file. The 2.5 days were spread out over a couple of weeks and three interactions with tech support including a promise to escalate that made just to close the call and never fulfilled. I will not feel secure about my financial data until this problem is corrected. I hope the problem is solved by Quicken because if it's solved by a new competitor entering the market, I will have waste more time maintaining two data files, while I switch over to the competitor.

  • SeaStar
    SeaStar Member ✭✭✭
    Options

    With Oct 2023 post, I'm glad to see efforts are underway to replace vague Validation summaries with specific data regarding fixes and non-resolved errors.