Backup or Copy File -> Create a Copy or Template aka "File Copy"

[Deleted User]

With one of the recent updates, the "File Copy" option has changed.  One obvious change is online services are removed in the copy file.

I have a question, that I don't see addressed in this forum.  This "File Copy" option is still being recommended as a file cleanup and troubleshooting tool.  Is the only difference between this version and the old version is that one would need to restore online services in the copy file?  The fact that all online services would need to be manually restored doesn't seem to be ever mentioned.

splasher

To be truthful about it, I don't think we know for sure if it is same or not.  The old copy was done at the transaction level and "rebuilt" the file, I don't remember any Quicken official telling us whether the new copy is at transaction level or just a copy like done by Windows Explorer.

Sherlock

A significant difference between the new copy and old copy is that new copy generates a new internal file identifier.  This means the new Quicken file will have it's own cloud account and may explain why Quicken chose to deactivate the Online Services in the new Quicken file.  You would not want to be issuing the same payments from each file.

Chris_QPW

I'm with @splasher in that they have never officially said anything (which is typical) and as such we have to guess.  But as a programmer I see no reason that they would really change the underline "copy".  What I see is that they had a problem, and they patch up the process to "fix it".

It has been reported many times that people have got their selves into trouble because they copied a Quicken data file and then used it as if it was a separate data file (as in used it for a template for another file).

In the past this worked fine because there wasn't any syncing to a Quicken Cloud dataset.  But now there is no matter what settings you have set.  How much does depend on what services you use like Express Web Connect, and settings like Sync to Mobile/Web, but there is always some syncing.

As @Sherlock pointed out each data file has a unique Id, which is used to connect it to a given Quicken Cloud dataset.  Now imagine two different Quicken data files syncing to the same Quicken Cloud dataset.  It can easily get messed up thinking that the user deleted an account in Quicken data file 1, when in reality they deleted it in data file 2.

This is a known problem they were facing.  So, for a long time we (the long time/SuperUsers) have been telling them they needed a way to change the unique Id in Quicken data file that is going to be used as a copy.  So, it isn't surprising at all that they did with the existing copy function.  But if you have just disconnected from the old Quicken Cloud dataset, then what does that mean for the online services that were connected to it?  Surely it is likely that it would cause problems if you left them connected.  So, the second part is perfectly logical too, as part of the operation you disconnect all the online services.

This fixes their problem, but of course causes another for the users that have been using the copy for the purposes of cleaning up data corruption, or just making a copy of their data file that isn't "template" for another data file.

splasher

To be truthful about it, I don't think we know for sure if it is same or not.  The old copy was done at the transaction level and "rebuilt" the file, I don't remember any Quicken official telling us whether the new copy is at transaction level or just a copy like done by Windows Explorer.

Sherlock

A significant difference between the new copy and old copy is that new copy generates a new internal file identifier.  This means the new Quicken file will have it's own cloud account and may explain why Quicken chose to deactivate the Online Services in the new Quicken file.  You would not want to be issuing the same payments from each file.

[Deleted User]

It seems to be still recommended for file repair and troubleshooting.  I guess that's my main question.  Is it still a good solution to fix file errors or corrupted file?  If it creates a new file identifier, will this cause a problem when someone uses the new identifier?

Chris_QPW

I'm with @splasher in that they have never officially said anything (which is typical) and as such we have to guess.  But as a programmer I see no reason that they would really change the underline "copy".  What I see is that they had a problem, and they patch up the process to "fix it".

It has been reported many times that people have got their selves into trouble because they copied a Quicken data file and then used it as if it was a separate data file (as in used it for a template for another file).

In the past this worked fine because there wasn't any syncing to a Quicken Cloud dataset.  But now there is no matter what settings you have set.  How much does depend on what services you use like Express Web Connect, and settings like Sync to Mobile/Web, but there is always some syncing.

As @Sherlock pointed out each data file has a unique Id, which is used to connect it to a given Quicken Cloud dataset.  Now imagine two different Quicken data files syncing to the same Quicken Cloud dataset.  It can easily get messed up thinking that the user deleted an account in Quicken data file 1, when in reality they deleted it in data file 2.

This is a known problem they were facing.  So, for a long time we (the long time/SuperUsers) have been telling them they needed a way to change the unique Id in Quicken data file that is going to be used as a copy.  So, it isn't surprising at all that they did with the existing copy function.  But if you have just disconnected from the old Quicken Cloud dataset, then what does that mean for the online services that were connected to it?  Surely it is likely that it would cause problems if you left them connected.  So, the second part is perfectly logical too, as part of the operation you disconnect all the online services.

This fixes their problem, but of course causes another for the users that have been using the copy for the purposes of cleaning up data corruption, or just making a copy of their data file that isn't "template" for another data file.

[Deleted User]

But strangely, it's the Quicken admin/moderators that have been recommending this new file copy for troubleshooting and as a fix for corrupted files, or for file issues.  

Chris_QPW

Damian said:

But strangely, it's the Quicken admin/moderators that have been recommending this new file copy for troubleshooting and as a fix for corrupted files, or for file issues.  

Not strange at all, as I was saying, it is my belief that the underline copy hasn't changed.  As such, that means the problems that were fixed by forcing a record-by-record copy would still be fixed using the new copy.  What's more they get an added troubleshooting step built in.  The very fact that it forces all the online services to be disconnected means that all of them will start fresh when the user has to reconnect them.

So, for their "troubleshooting" they get two big hammers at the same time.

Royal pain for the user, easier for them.

[Deleted User]

Ok Thanks @Chris_QPW.  Your explanations are always helpful and interesting.  

BK

@Chris_QPW , all,

Question given a new identifier:

Many users would probably use the copy feature to repair some issues, and it is a reasonable scenario that they would prefer to rename it back to the original file name (delete the old file).  So what is the best practice to avoid any potential conflicts?

My thinking is this: I make a Quicken-copy of BK_Data > BK_DataCopy, and then I rename it back to BK_Data.  Should I delete the associated cloud datasets first before opening the new file, allowing it to create a fresh one?  (I don't want to test this to avoid the future dataset exceeded message)

Chris_QPW

BK said:

@Chris_QPW , all,

Question given a new identifier:

Many users would probably use the copy feature to repair some issues, and it is a reasonable scenario that they would prefer to rename it back to the original file name (delete the old file).  So what is the best practice to avoid any potential conflicts?

My thinking is this: I make a Quicken-copy of BK_Data > BK_DataCopy, and then I rename it back to BK_Data.  Should I delete the associated cloud datasets first before opening the new file, allowing it to create a fresh one?  (I don't want to test this to avoid the future dataset exceeded message)

I'm a gluten for punishment  so I went ahead and tested this.

As I suspected Quicken can handle this situation without the user doing anything since the unique Id is different it knows to keep them separate.

Here is an example of "doing nothing":


As you can see it just put (1) at the end of the Cloud Account Name when it found a conflict in the name.
One approach to clean it up at this point would be to delete the old one, and then edit the new one to remove the (1). Note that removing the (1) isn't really needed, but it is nice to keep the same name as your data file.

I also tried it with this flow.
Create copy.
Go into data file connected to cloud to delete the Cloud dataset of the old data file.
Rename data files so that the new copy is using the old data file's name.
Open this new copy of the data file and setup one of my accounts and update (Note I'm using Direct Connect for everything so I'm less likely to have any "server problems".).
Switch to cloud connected data file to check out what happened.
The new cloud dataset name is the same as the old/current data file (no (1) appended).

Note this isn't an exhaustive test since I don't have any Express Web Connect accounts, and certainly not and Express Web Connect + accounts, but it does all seem to work properly no matter which way you go.

Sherlock

BK said:

@Chris_QPW , all,

Question given a new identifier:

Many users would probably use the copy feature to repair some issues, and it is a reasonable scenario that they would prefer to rename it back to the original file name (delete the old file).  So what is the best practice to avoid any potential conflicts?

My thinking is this: I make a Quicken-copy of BK_Data > BK_DataCopy, and then I rename it back to BK_Data.  Should I delete the associated cloud datasets first before opening the new file, allowing it to create a fresh one?  (I don't want to test this to avoid the future dataset exceeded message)

The conflict you want to avoid is using the same file identifier for distinct active Quicken files.  In other words, you do not want to use a Windows copy or a restored Quicken file backup as a new working Quicken file you will be using along side the original.

Note:  I suspect using Quicken files with the same file identifiers but associated with distinct Quicken IDs may be acceptable.

Chris_QPW

BTW the adding of (1) to the name is actually new to me.  I have seen in the past that Quicken would happily have multiple Quicken Cloud datasets with the same name.  I think they just added that to get rid of some of the confusion for the users.  The underline way the connection is made between the data file and the Quicken Cloud dataset is the unique Id (and the Quicken Id of course).

Chris_QPW

I think there is some confusion on what triggers the "too many dataset" error.

I can have a hundred data files, and not get that error.
What triggers the error are "unlinked/hidden" datasets.

As in, you do a restore from a backup.  The restore process creates a new copy or your dataset, and "hides" the old one.  You now have two copies of the same dataset.  Repeat and this is how you get more of them and hit the limit.

I have also seen that error thrown for situations where there is no way that there can be too many (a new install of Quicken 2017 with a new data file for instance).  In this case I'm guessing they "guess" that some other kind of dataset error is an "exceed error" and report it incorrectly.

Sherlock

Chris_QPW said:

I think there is some confusion on what triggers the "too many dataset" error.

I can have a hundred data files, and not get that error.
What triggers the error are "unlinked/hidden" datasets.

As in, you do a restore from a backup.  The restore process creates a new copy or your dataset, and "hides" the old one.  You now have two copies of the same dataset.  Repeat and this is how you get more of them and hit the limit.

I have also seen that error thrown for situations where there is no way that there can be too many (a new install of Quicken 2017 with a new data file for instance).  In this case I'm guessing they "guess" that some other kind of dataset error is an "exceed error" and report it incorrectly.

Quicken is being intentionally vague with regard to the "too many dataset" error message.  Suffice it to say, the error message has no relevance in this discussion.

BK

Thank you both.  The new addition of (1) in the dataset name is also first I have seen, and validates what I was thinking as well.

FYI, after reading one of the very old & long discussions on "restore vs rename" a backup file, I too decided to use rename from now on to avoid the additional dataset creation (assuming that thinking is still valid).

Chris_QPW

For what it is worth, I have never used Quicken's backup/restore feature.  I have my own system for backing up important files like Quicken's data file, and a "restore" for me is just replacing my data file with a copy.  Outside of testing, I have never had a problem with doing it that way (no "exceeded dataset" kinds of errors).

Sherlock

Chris_QPW said:

For what it is worth, I have never used Quicken's backup/restore feature.  I have my own system for backing up important files like Quicken's data file, and a "restore" for me is just replacing my data file with a copy.  Outside of testing, I have never had a problem with doing it that way (no "exceeded dataset" kinds of errors).

We always use Quicken's backup/restore feature frequently.  We use Duplicati for backing up important files to cloud storage.  We don't have any kind of errors.

Chris_QPW

Sherlock said:

Chris_QPW said:

For what it is worth, I have never used Quicken's backup/restore feature.  I have my own system for backing up important files like Quicken's data file, and a "restore" for me is just replacing my data file with a copy.  Outside of testing, I have never had a problem with doing it that way (no "exceeded dataset" kinds of errors).

We always use Quicken's backup/restore feature frequently.  We use Duplicati for backing up important files to cloud storage.  We don't have any kind of errors.

If you use the restore function frequently, one has to wonder if what Quicken Inc has been telling us about how it works is true.  Given that I don't use it, I can only go by what is stated in this forum.

Sherlock

Chris_QPW said:

Sherlock said:

Chris_QPW said:

For what it is worth, I have never used Quicken's backup/restore feature.  I have my own system for backing up important files like Quicken's data file, and a "restore" for me is just replacing my data file with a copy.  Outside of testing, I have never had a problem with doing it that way (no "exceeded dataset" kinds of errors).

We always use Quicken's backup/restore feature frequently.  We use Duplicati for backing up important files to cloud storage.  We don't have any kind of errors.

If you use the restore function frequently, one has to wonder if what Quicken Inc has been telling us about how it works is true.  Given that I don't use it, I can only go by what is stated in this forum.

Again, Quicken Inc is being intentionally vague.