Update cached data from Schwab/other investment FIs more frequently [Edited]

veegee

I am an active trader and have 20 to 50 stock transactions in a given day. I used to be able to download my transactions multiple times a day. I usually did about 4 to 5 downloads, spread through the day and reconcile the transactions thus far during the day.

I can no longer do this. [Removed - Language] Yes, Quicken will permit the downloads, but the data fetches do NOT go all the way to Schwab. Studying the pattern of the data downloaded, I believe, with fair certainty, that the data fetches from Schwab are "cached" (temporarily stored) at an intermediate point somewhere. After the initial caching, further downloads will result in the data coming from the cache and not from Schwab. This happens till the 'cache expiry' time is reached, which appears to be around about a half day to a full day.

Issue started with EWC deployment IMHO. And I suspect this caching is occurring at the data aggregator (Intuit), but not sure of it.

This post is an enhancement request to reduce the cache interval to an hour or less. Assuming the caching is done to reduce the load on the servers (either Schwab's or the data aggregator's) or to protect against DOS attacks, changing to an hour or so will have practically no impact. This is because, IMO, very few people actually do many downloads per day. The change will greatly help those of us who do.

I believe the issue exists with all brokerage institutions and not just Schwab.

I expect that the code change required should just be changing the timeout number (in the code or database) and should be a very easy thing to implement/test IMHO.

[Quicken Deluxe on Windows]

Quicken Kristina

Hello @veegee,

Since you posted in another discussion that this is an enhancement request, I converted this discussion to an Idea post and moved it to the Product Ideas section of the Community. I also edited the title to more closely reflect the change you're asking for.

Thank you!

veegee

Thank you, @Quicken Kristina !

Perhaps I should have reported it as a bug (?) (It used to work fine, so to me it is actually a bug).

q_lurker

@veegee It has often been stated in this site by long-time users that Quicken is not a good tool for the active day-trader. They do not fall within Quicken’s apparent target demographic. This idea / post is yet another point making that case.

Which is a way of saying (my opinion), the program is operating as designed. This is not a bug.

Good luck with your idea.

veegee

Yes, I have seen that statement, @q_lurker, but with all due respect, I hope you don't mind that I do not agree with that. I have been using Quicken for 20+years and while it is not exactly the fastest at dealing with lots of transactions, it can very easily be sufficient to handle things. With minor tweaks it can be made to be rated good, even. Active day-trader scenario can just be looked at as a stress test on the software. All improvements made also help all other users. I am quite sure everyone expects a 'download initiated by a user' to fetch all up-to-date data. It is that fact that it does not that makes me want to say it is a bug. But, let's leave it at that.

I agree with you, though - from what I have seen, this is unlikely to get addressed. Sad.

This is my humble opinion (and I have been in the software industry for decades, with all of development, management and marketing hats on).

q_lurker

Thanks for the positive feedback from an active day-trader.

I am quite sure everyone expects a 'download initiated by a user' to fetch all up-to-date data.

I am quite sure expectations of everyone are never the same. I deal with one FI who I know does not provide same day transactions until sometime past the end of business that day.

Jim_Harman

Here is an earlier discussion with some data about the timing of Schwab updates.

https://community.quicken.com/discussion/7921038/quicken-throttling-transactions-from-charles-schwab

Rocket J Squirrel

@veegee Have you considered moving your holdings to a brokerage that supports Quicken's Direct Connect? "Express" means daily and "Direct" might get you closer to realtime data. May be worth the experiment to move one or more active securities to a new brokerage.

Rick8

@Rocket J. Squirrel What brokerage firms support Direct Connect? DC works all the time. Never had issues. When Schwab changed last year, it was a total disaster for 6 months! I'm sure you remember.

bmciance

Fidelity still supports direct connect BUT, their data is not real time. Entries made today show up as "pending" and are not downloaded until tomorrow.

veegee

@Rocket J Squirrel , I am afraid there are far more important factors to consider when choosing a brokerage for day trading than whether they can do Direct Connect to Quicken. Certainly at least for me. I have Fidelity as well as Schwab, and, for reasons I do not want to go into here, I would never do active trading on Fidelity. Now that Schwab has ThinkOrSwim, I would not consider switching at all barring a total meltdown.

As @bmciance points out, Fidelity does not do real-time, but seems to update at end-of-day or beginning of day or midnight.

@Rick8 , I remember those fiasco days (months). Wish things to go back to how it was before that, but for now things have somewhat stabilized to a somewhat manageable degree. Except for this pain-in-the-[whatever] limitation on how often one can download. (I don't necessarily want 'download any time' but at least let us update a few times a day).

@Jim_Harman, thank you for the pointer to the other thread. I will read and connect to that as well. ( Update: that thread is closed now :-( )

Chris_QPW

personally have found the cashing to be a little bit of a pain, but one has to wonder7 where the motivation for doing that comes from. I know that the data no matter whether it's direct connect or Express has always been throttled by either the data being available on the financial institution or policies that the financial institutions have stated to Intuit as part of the agreement for doing the downloads. In other words the financial institutions might have already stated that they don't want downloads be more frequent than a certain amount of time. Given given how secretive Quicken Inc and Intuit are about this kind of stuff I doubt we'll ever know where the actual limiting factor is. But one thing that is definitely always been true is they have never promised real time or anything close to it.

veegee

@Chris_QPW I can think of only two things for the throttling. a) Preventing overload of Schwab servers or Intuit servers due to excessive data fetches by customers, and b) Preventing Denial-of-Service attacks on Schwab servers. Others can perhaps think of more reasons.

Schwab makes data available for download on their web servers all the time in real-time. So do other FIs, including Fidelity. However, trades are subject some 'final adjustments' at the end of the day, and this might be the reason some FIs such as Fidelity wait till the end of the day - so they can provide only the final data. Schwab also, in some cases, corrects the data at the end of the day - they simply replace the data. So it is possible to get duplicate entries for a given transaction. This does not happen often, though.

Let us assume that the overload concerns are valid. However, the current level of throttling is a HUGE overkill in my opinion. Most likely, IMO, they just pulled a number off the air with no real data on the load on the servers with different timeout numbers. IMHO, they could very easily reduce the cache timeout to something less than an hour, and they will notice no difference. (Personally I think even 10 minutes will be just fine.)

The change is such an easy one to make and requires little testing. It is definitely worth considering the change. Probably Intuit has to make the change. (We are all caught in this love trianglenow.)

Another way to implement this throttling would be to keep a counter of the number of downloads and limit it to a maximum of, say five or 10 per day. Requires more code change and testing.

Sometimes, in software development, one has to periodically give a higher priority to a change because it is a simple change or low risk change. Otherwise the overall quality will suffer - not from major issues, but a large number of minor issues. (Death by a thousand cuts.)

Chris_QPW

@veegee One has to understand that the policies that a given financial institution has for their website doesn't necessarily reflect what they might want for third party software. But with that being said, after I have thought about it some more, in my opinion this caching was done the Quicken Inc or maybe the Intuit servers. It seems to be a fixed amount somewhere in the "few hours" range, which implies it is for all financial institutions, and as such not the policy of a given financial institution.

veegee

@Chris_QPW I agree with you about the policies.

I was merely pointing out that IF Schwab's concern was overload/DOS on Schwab's data servers, then those concerns apply to their web servers as well. There are other industry practices to mitigate the overload/DOS concerns internally, without telling your customers 'hey, access these pages only twice a day'.

Of course, we do not know where the restriction is coming from. Is it imposed by Schwab? Or a group of FIs? Or self-imposed by Intuit for whatever reason? Who knows?

All I can say is that I really doubt if anything bad will happen if they reduced the throttling to allowing once an hour or less. Possible, but doubtful. I just think they overkilled it. My opinion comes from having had a lot of experience dealing with software/system performance at a major company.

veegee

Note: I am trying to get some traction on this issue by means other than this forum as well.

Chris_QPW

@veegee what I do know for a fact is that there were some financial institutions that restricted Quicken access to once a day even though they certainly wouldn't have had such a restriction on their website. So, the policy can be different, it isn't a given that they are the same. I doubt much of that exists these days, but it did happen.

I agree that the caching is too restrictive, and that I doubt that reducing it would cause any real problems. I have a feeling this was more for Quicken Inc than anyone else.

veegee

@Chris_QPW 👍