Cardboard Posted February 2, 2018 Share Posted February 2, 2018 I am now having trouble logging and getting quotes from my Canadian trading platform. For weeks, Canadian banks have been apparently trying to fix/patch problems due to high trading volume on pot stocks and apologizing to their customers!!! Never liable however. You gotta be kidding me! How is it that the same platforms were "capable" to handle Nortel Networks volumes and all the other stocks in that Internet bubble era and that 18 years later, the platforms are unable to handle the volume from a handful of companies? 18 years has to be an eternity in the world of software development... It is a complete joke! Cardboard Link to comment Share on other sites More sharing options...
StubbleJumper Posted February 2, 2018 Share Posted February 2, 2018 I am now having trouble logging and getting quotes from my Canadian trading platform. For weeks, Canadian banks have been apparently trying to fix/patch problems due to high trading volume on pot stocks and apologizing to their customers!!! Never liable however. You gotta be kidding me! How is it that the same platforms were "capable" to handle Nortel Networks volumes and all the other stocks in that Internet bubble era and that 18 years later, the platforms are unable to handle the volume from a handful of companies? 18 years has to be an eternity in the world of software development... It is a complete joke! Cardboard Part of the problem is that trades today cost less in nominal dollars than they cost 18 years ago. Back in 2001, a trade with TD or RBCDI or I*Trade was $29. Today it's $10 with those companies and $7 with CIBC. In real, inflation adjusted, dollars, a trade today is about $5 compared to the $29 of 2001. What that means is that today's 23 year-old "investors" who are chasing the hottest thing face very low costs to trade in and out of weed stocks. While a young guy in 2001 might have bought and sold Nortel once per week, the low cost of commissions means the the young Turks of today can trade their weed stocks every day or even several times per day without incurring much of a frictional cost. These young studs are the ones who have blown up the banks' trading platforms. What this really underlines to me is the importance of a preferred client group such as TD's President's Club, or RBCDI's Gold Circle. The banks very badly need to segment their clientele to ensure that they are not pissing off the small group that might have $1m+ of assets while accommodating the new generation who are rapidly churning their $10k RRSP. I'm not sure that any of the banks have figured out how to effectively do this. SJ Link to comment Share on other sites More sharing options...
rb Posted February 2, 2018 Share Posted February 2, 2018 Well first off it's not like the bank systems were that good in the past. I remember one day when Nortel earning were out I ended 3 times my position in a different stock because the Nortel's volume overwhelmed TD's systems and they stopped sending out trade confirmations. I lost a lot of money that day because of this. Fortunately there is a simple solution to all of this: Interactive Brokers. Link to comment Share on other sites More sharing options...
KCLarkin Posted February 2, 2018 Share Posted February 2, 2018 How is it that the same platforms were "capable" to handle Nortel Networks volumes and all the other stocks in that Internet bubble era and that 18 years later, the platforms are unable to handle the volume from a handful of companies? 18 years has to be an eternity in the world of software development... The obvious answer is that they are using 18 year old systems. With 18 years of patches, hacks, and modifications... IBKR is the solution. Disclosure: Long IBKR. I haven't noticed any major issues with BMO IL but client response times are crazy right now. One of the benefits of the five star program is supposed to be enhanced customer responsiveness. I am slowly shifting assets over to IBKR. Link to comment Share on other sites More sharing options...
Viking Posted February 2, 2018 Share Posted February 2, 2018 Cardboard, thank you for getting the discussion going on this. Here was my experience. Back in Dec when the US Federal Government said it was not supportive of legalizing pot (or something like that) there was a panic with pot stocks in Canada. The surge in trading that resulted caused RBC’s trading platform (used by those with self directed accounts) to fail. For much of the day it was not possible to log in to your account execute a trade. When the online trading account is not working, you are to call in to execute your trade. The problem is the phone lines were swamped and the wait times to get through were +4 hours. For high net worth customers (for RBC Royal Circle that is customers with more than $250,000 in assets), RBC has a separate phone number that is supposed to provide quicker service. This line also had wait times of +4 hours. The message simply said something to the effect, “we are experiencing higher than normal call volume and we will get to your as soon as we can”. It did not communicate a rough wait time. So basically investors were unable to trade online or contact anyone to execute a trade for much of the trading day. The system did get back up and running late in the trading day. All of this was caused by a spike in volume of pot stocks. I was not impacted personally by what happened that day as I do not own any of the pot stocks. I was lucky. However, it did alert me to some serious issues that I needed to problem solve. 1.) what do I do the next time their online system crashes? At some point we are going to get a real sell off in the market. If something as small as pot stocks can cause the system to crash, what happens when a real sell off happens and trading volume spikes, likely on multiple days? 2.) what do I do when the phone lines get overwhelmed? 3.) when 1 and 2 above fail what can I do to execute a trade? I called RBC a week later and was told there is nothing I can do. I simply have to wait until their either fix their online system or a phone line is freed up. And the answer is the same if you are a high net worth individual. I asked if it was not possible to go into one of the bank branches and be able to execute a trade there and the answer was a definite no. I told them that this was a very important issue to me and if they were not able to provide a solution I may need to move all of my investments. The person said they record all calls and she would review my call at their weekly meeting. I have received no call back. I called BMO recently and reviewed the situation with them. They had the same issues as RBC I was told. Their online system crashed. And their phone lines had wait times of multiple hours. And, just like RBC, they have no solution when the same thing happens again in the future (as I am sure it will). Just like when investing, I try and be forward looking when managing things that are important. The time to fix something is not when you are desperate. Solving this issue is going to take a little more digging :-) Does anyone out there have a solution they are happy with? PS: I told both RBC and BMO that if a company can find a solution to this issue they will be well positioned to pick up lots of new business (as surely I cannot be the only investor who feels it is a very important issue). Link to comment Share on other sites More sharing options...
StubbleJumper Posted February 2, 2018 Share Posted February 2, 2018 How is Interactive Brokers the solution? I understand that their platform was not overwhelmed last month, which gives them a gold star compared to the big banks. But, what has Interactive Brokers done which makes their computer trading platform scaleable for these types of events? Do they routinely maintain 20X their normal capacity as a risk management tool for the high volume days? SJ Link to comment Share on other sites More sharing options...
rb Posted February 2, 2018 Share Posted February 2, 2018 As far as I know IB's platform has never been overwhelmed by volume and we've been through some hairy times in the past. I don't know what spare capacity they keep. But their track record points to them as a good solution. Link to comment Share on other sites More sharing options...
Uccmal Posted February 3, 2018 Share Posted February 3, 2018 Cardboard, thank you for getting the discussion going on this. Here was my experience. Back in Dec when the US Federal Government said it was not supportive of legalizing pot (or something like that) there was a panic with pot stocks in Canada. The surge in trading that resulted caused RBC’s trading platform (used by those with self directed accounts) to fail. For much of the day it was not possible to log in to your account execute a trade. When the online trading account is not working, you are to call in to execute your trade. The problem is the phone lines were swamped and the wait times to get through were +4 hours. For high net worth customers (for RBC Royal Circle that is customers with more than $250,000 in assets), RBC has a separate phone number that is supposed to provide quicker service. This line also had wait times of +4 hours. The message simply said something to the effect, “we are experiencing higher than normal call volume and we will get to your as soon as we can”. It did not communicate a rough wait time. So basically investors were unable to trade online or contact anyone to execute a trade for much of the trading day. The system did get back up and running late in the trading day. All of this was caused by a spike in volume of pot stocks. I was not impacted personally by what happened that day as I do not own any of the pot stocks. I was lucky. However, it did alert me to some serious issues that I needed to problem solve. 1.) what do I do the next time their online system crashes? At some point we are going to get a real sell off in the market. If something as small as pot stocks can cause the system to crash, what happens when a real sell off happens and trading volume spikes, likely on multiple days? 2.) what do I do when the phone lines get overwhelmed? 3.) when 1 and 2 above fail what can I do to execute a trade? I called RBC a week later and was told there is nothing I can do. I simply have to wait until their either fix their online system or a phone line is freed up. And the answer is the same if you are a high net worth individual. I asked if it was not possible to go into one of the bank branches and be able to execute a trade there and the answer was a definite no. I told them that this was a very important issue to me and if they were not able to provide a solution I may need to move all of my investments. The person said they record all calls and she would review my call at their weekly meeting. I have received no call back. I called BMO recently and reviewed the situation with them. They had the same issues as RBC I was told. Their online system crashed. And their phone lines had wait times of multiple hours. And, just like RBC, they have no solution when the same thing happens again in the future (as I am sure it will). Just like when investing, I try and be forward looking when managing things that are important. The time to fix something is not when you are desperate. Solving this issue is going to take a little more digging :-) Does anyone out there have a solution they are happy with? PS: I told both RBC and BMO that if a company can find a solution to this issue they will be well positioned to pick up lots of new business (as surely I cannot be the only investor who feels it is a very important issue). It is an important issue. 2 or 3 weeks ago. TD was done for quite a few hours 3 or 4 days running. I complained and got an apology, and the usual BS speil. Today & yesterday would have been really lousy days not to have access. I didn't do anything much but you never know. Link to comment Share on other sites More sharing options...
beerbaron Posted February 3, 2018 Share Posted February 3, 2018 I did not trade in the last month but anybody knows how did Questrade support the spike in trades? IB could be running on AWS backbone that scales the CPU power based on demand. No need to maintain a constant 20x capacity. BeerBaron Link to comment Share on other sites More sharing options...
Spekulatius Posted February 3, 2018 Share Posted February 3, 2018 How is Interactive Brokers the solution? I understand that their platform was not overwhelmed last month, which gives them a gold star compared to the big banks. But, what has Interactive Brokers done which makes their computer trading platform scaleable for these types of events? Do they routinely maintain 20X their normal capacity as a risk management tool for the high volume days? SJ In my experience, Ib worked, when nothing else would (Fidelity, E*TRADE, Wells Fargo ) during October 2018, the flash crashes etc. It is the most reliable platform I am aware of. Link to comment Share on other sites More sharing options...
Cardboard Posted February 3, 2018 Author Share Posted February 3, 2018 That is the thing I don't get with this latest excuse: I never had a problem in 2008/2009. Talk about liquidity events! And now you have a few stocks such as ACB and WEED with around 100,000 trades/day each and everything comes crashing down? In an up move for these stocks? There was a minor lock-up today but, it has cleared in a few minutes in a really nasty market for these stocks. So they must have made some changes. Still crazy. At the very least, halt trading in the ones causing issues. Cardboard Link to comment Share on other sites More sharing options...
StubbleJumper Posted February 3, 2018 Share Posted February 3, 2018 I did not trade in the last month but anybody knows how did Questrade support the spike in trades? IB could be running on AWS backbone that scales the CPU power based on demand. No need to maintain a constant 20x capacity. BeerBaron Well, that's a good possibility. So, perhaps the big banks are running in-house systems which are not scaleable and thus fail when demand spikes by a factor of X, while IB has outsourced to a scaleable service provider. That would be a good reason to consider IB. Not too sure how I feel about the security implications. SJ Link to comment Share on other sites More sharing options...
rb Posted February 3, 2018 Share Posted February 3, 2018 As far as I know IB services are in house. They don't use any cloud service. This is not that complicated. IB built a proper operation. The banks also have reliable operations that don't go down... on the institutional side. The systems on the retail side are just shit. That's because they didn't care to build a good operation. As I said this isn't that complicated. Also if you choose to stay on the shit platform then you don't really have a right to complain. Edit: I also have some accounts at Questrade... and they're crap as well. But I don't complain because I know they're crap and made a conscious decision to have those accounts there. Link to comment Share on other sites More sharing options...
StubbleJumper Posted February 3, 2018 Share Posted February 3, 2018 This is not that complicated. IB built a proper operation. The banks also have reliable operations that don't go down... on the institutional side. The systems on the retail side are just shit. That's because they didn't care to build a good operation. As I said this isn't that complicated. Also if you choose to stay on the shit platform then you don't really have a right to complain. rb, I am a little curious how you know this. Up until two months ago, there were historically essentially zero capacity related IT failures amongst the major Canadian brokers. We seemed to have experienced a few days of wildly excessive demand which resulted in some of their platforms failing while others appear to be business as usual. Do you have some sort of industry IT knowledge that one broker built a good or proper system and one simply cobbled together some spare hardware and slapped together a bit of code? That situation is quite clearly a possibility, but without actually having worked in one of the brokerages, it's not clear to me how one would obtain that knowledge. I ask this because I am genuinely curious about why some brokers failed to deliver the goods while it was business as usual for others. In December, it seems clear that some of the brokers fell short on capacity -- did they have 80% or 90% of the capacity needed to manage the spike in pot stock trading, or did they have 60% or 70% of the required capacity for that demand spike? IB seems to have had at least 100% of the needed capacity, but was there still some spare capacity? Could IB have handled 50% more volume, would the platform take double the volume in December? Can we instead attribute IB's success to the fact that they might not have a legion of 20 year-olds who vigorously trade pot stocks because most of their clients are more serious, experienced investors? I am quite curious about all of this because the next generalized market event could quite possibly involve considerably higher volume than what we saw in December. I don't generally panic and sell when the market goes down, but it would be nice to at least be able sell.... SJ Link to comment Share on other sites More sharing options...
EliG Posted February 3, 2018 Share Posted February 3, 2018 My understanding is that bank brokerages got swamped by a flood of new small gamblers clients who opened their first account to trade pot stocks or bitcoin. They required a lot of hand holding due to lack of experience; hence long wait lines on the phones. IB Canada is not a natural choice for this crowd. I think it's possible that IB didn't need to scale up because their web traffic and trading volumes remained relatively normal. I don't have any data to back this up. Link to comment Share on other sites More sharing options...
KCLarkin Posted February 3, 2018 Share Posted February 3, 2018 One obvious answer is that IBKR is a global operation with significantly larger scale than any of the Canadian banks. So a few retail traders day trading pot stocks won’t bring the system down. But it also has significantly more experience scaling its system. With a properly designed system, it isn’t hard to quickly scale capacity. But all of the big Canadian banks are relatively small brokers. Anyway, the experience is that IBKR has been stable when other platforms went down (eg during flash crash). Reasonable expectation is that it is your best bet. But no guarantees that any platform be stable in a real panic. Remember brokers just stopped answering the phone in the 87 crash. Link to comment Share on other sites More sharing options...
StubbleJumper Posted February 3, 2018 Share Posted February 3, 2018 One obvious answer is that IBKR is a global operation with significantly larger scale than any of the Canadian banks. So a few retail traders day trading pot stocks won’t bring the system down. But it also has significantly more experience scaling its system. With a properly designed system, it isn’t hard to quickly scale capacity. But all of the big Canadian banks are relatively small brokers. Anyway, the experience is that IBKR has been stable when other platforms went down (eg during flash crash). Reasonable expectation is that it is your best bet. But no guarantees that any platform be stable in a real panic. Remember brokers just stopped answering the phone in the 87 crash. Now that's an interesting possibility that I hadn't considered. IB might have a single mega data centre located in, say, New Jersey, which services all of their clients around the world. I had always assumed that they would use de-centralized data centres, but I suppose that there's no real reason from a regulatory perspective to use a Canadian data centre for Canadian clients and a US data centre for US clients, etc. It's always good to challenge our assumptions! SJ Link to comment Share on other sites More sharing options...
rb Posted February 3, 2018 Share Posted February 3, 2018 This is not that complicated. IB built a proper operation. The banks also have reliable operations that don't go down... on the institutional side. The systems on the retail side are just shit. That's because they didn't care to build a good operation. As I said this isn't that complicated. Also if you choose to stay on the shit platform then you don't really have a right to complain. rb, I am a little curious how you know this. Up until two months ago, there were historically essentially zero capacity related IT failures amongst the major Canadian brokers. We seemed to have experienced a few days of wildly excessive demand which resulted in some of their platforms failing while others appear to be business as usual. Do you have some sort of industry IT knowledge that one broker built a good or proper system and one simply cobbled together some spare hardware and slapped together a bit of code? That situation is quite clearly a possibility, but without actually having worked in one of the brokerages, it's not clear to me how one would obtain that knowledge. I ask this because I am genuinely curious about why some brokers failed to deliver the goods while it was business as usual for others. In December, it seems clear that some of the brokers fell short on capacity -- did they have 80% or 90% of the capacity needed to manage the spike in pot stock trading, or did they have 60% or 70% of the required capacity for that demand spike? IB seems to have had at least 100% of the needed capacity, but was there still some spare capacity? Could IB have handled 50% more volume, would the platform take double the volume in December? Can we instead attribute IB's success to the fact that they might not have a legion of 20 year-olds who vigorously trade pot stocks because most of their clients are more serious, experienced investors? I am quite curious about all of this because the next generalized market event could quite possibly involve considerably higher volume than what we saw in December. I don't generally panic and sell when the market goes down, but it would be nice to at least be able sell.... SJ I did work for a couple of the banks (not for the online divisions though) and I know that they were woefully under invested in their online brokerages. They're kind of a joke. The feeling is that whatever it is it's good enough for the retail investors. If sometimes they don't work that's just life. I also think that you're thinking about this capacity issue the wrong way. I'm not a tech guy (so don;t take me as an authority) but I know a bit about it. You can generally process huge amounts of traffic/data at data center level with relatively little equipment if it's well configured. But IB and the banks brokers are fundamentally different in how they run. With IB you use TWS which is stand alone on your computer and communicates with IBs servers using data streams over certain communications protocols. This is a (ralatively) simple and robust solution. Going about it this way IB may very well have the capacity to do 10 or 20 times their normal volume because that extra capacity would not be very expensive. The banks on the other hand use a web application to deliver the service. These apps are probably not very good to start - none of the bank tech is. It's also a much more complicated solution than what IB uses. Put it under stress and it craps out. You can't know for sure what happened unless you were there but if I were to bet, I would bet that it was user application that crapped out and not their trading engine in the back because it ran out of capacity. Oh and regarding a market crisis, they're guaranteed to not work. They weren't working on volatile days in 2015 when we had that pullback and those were not that bad historically speaking. Link to comment Share on other sites More sharing options...
StubbleJumper Posted February 3, 2018 Share Posted February 3, 2018 This is not that complicated. IB built a proper operation. The banks also have reliable operations that don't go down... on the institutional side. The systems on the retail side are just shit. That's because they didn't care to build a good operation. As I said this isn't that complicated. Also if you choose to stay on the shit platform then you don't really have a right to complain. rb, I am a little curious how you know this. Up until two months ago, there were historically essentially zero capacity related IT failures amongst the major Canadian brokers. We seemed to have experienced a few days of wildly excessive demand which resulted in some of their platforms failing while others appear to be business as usual. Do you have some sort of industry IT knowledge that one broker built a good or proper system and one simply cobbled together some spare hardware and slapped together a bit of code? That situation is quite clearly a possibility, but without actually having worked in one of the brokerages, it's not clear to me how one would obtain that knowledge. I ask this because I am genuinely curious about why some brokers failed to deliver the goods while it was business as usual for others. In December, it seems clear that some of the brokers fell short on capacity -- did they have 80% or 90% of the capacity needed to manage the spike in pot stock trading, or did they have 60% or 70% of the required capacity for that demand spike? IB seems to have had at least 100% of the needed capacity, but was there still some spare capacity? Could IB have handled 50% more volume, would the platform take double the volume in December? Can we instead attribute IB's success to the fact that they might not have a legion of 20 year-olds who vigorously trade pot stocks because most of their clients are more serious, experienced investors? I am quite curious about all of this because the next generalized market event could quite possibly involve considerably higher volume than what we saw in December. I don't generally panic and sell when the market goes down, but it would be nice to at least be able sell.... SJ I did work for a couple of the banks (not for the online divisions though) and I know that they were woefully under invested in their online brokerages. They're kind of a joke. The feeling is that whatever it is it's good enough for the retail investors. If sometimes they don't work that's just life. I also think that you're thinking about this capacity issue the wrong way. I'm not a tech guy (so don;t take me as an authority) but I know a bit about it. You can generally process huge amounts of traffic/data at data center level with relatively little equipment if it's well configured. But IB and the banks brokers are fundamentally different in how they run. With IB you use TWS which is stand alone on your computer and communicates with IBs servers using data streams over certain communications protocols. This is a (ralatively) simple and robust solution. Going about it this way IB may very well have the capacity to do 10 or 20 times their normal volume because that extra capacity would not be very expensive. The banks on the other hand use a web application to deliver the service. These apps are probably not very good to start - none of the bank tech is. It's also a much more complicated solution than what IB uses. Put it under stress and it craps out. You can't know for sure what happened unless you were there but if I were to bet, I would bet that it was user application that crapped out and not their trading engine in the back because it ran out of capacity. Oh and regarding a market crisis, they're guaranteed to not work. They weren't working on volatile days in 2015 when we had that pullback and those were not that bad historically speaking. Thanks for the insight. It's very helpful. It's entirely possible that the banks' web-server was a weak link, but the back end might have been fine. That makes perfect sense that a stand-alone app would work better than a web-based platform in that circumstance. I've never been too impressed by the brokers' websites, but from a user perspective it's all a black box, so it's helpful to get insight from somebody who has at least gone for a beer with people who work on that stuff. SJ Link to comment Share on other sites More sharing options...
DavidVY Posted February 3, 2018 Share Posted February 3, 2018 IB actually provides back-end service/market routing for other brokerages. Very likely they will be the backbone to all e-trading eventually just from a cost basis. Front office/back office separation Link to comment Share on other sites More sharing options...
rb Posted February 6, 2018 Share Posted February 6, 2018 Does anyone have anything to report regarding how various brokers behaved today? Link to comment Share on other sites More sharing options...
Cardboard Posted February 6, 2018 Author Share Posted February 6, 2018 Zero problem whatsoever. One of the big 5. Cardboard Link to comment Share on other sites More sharing options...
nodnub Posted February 6, 2018 Share Posted February 6, 2018 IBKR seems to have an approach of continual automation+improvement and elimination of issues that waste their staff resources. I've read in the past that IBKR is extremely focused on automating any process that required human interaction. In comparison, the big 5 banks seem to block automation -- they have required me to make phone calls to execute trades in slightly less common securities, or to handle things like USD denominated TSX listed stocks showing up correctly. If they can't get pretty simple things right without a phone call... that adds a lot of delays and overhead. It looks like some of the Big 5 brokers offer trading access through a standalone application that runs under Windows (similar to IB's TWS - Trader Workstation) BMO IL has "Market Pro" software. Looks like you need to be Gold Star account to access it. Scotia iTrade has "FlightDesk" software TD InvestDirect has "Advanced Dashboard" software All these platforms looks like the same Windows Java-based program, probably white-labelled software solution. Did anyone use these kind of platforms during the time periods in question? Was there problem with Trade execution from these platforms? Link to comment Share on other sites More sharing options...
Cardboard Posted February 6, 2018 Author Share Posted February 6, 2018 If you cannot log at all into the primary account, then you cannot use the fancy trading platform. That is unless there is a way to access it directly which I don`t know how. Cardboard Link to comment Share on other sites More sharing options...
KCLarkin Posted February 6, 2018 Share Posted February 6, 2018 Does anyone have anything to report regarding how various brokers behaved today? There was some reports of outages at a couple U.S. brokerages. Edit to add this thread: Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now