Finding a cloud for a pharmacy chain. The story of how our client nearly lost $175 000

Clouds belong to that type of modern technology that can be both scary and intriguing. The reason behind this is that not many users are willing to share their experiences with cloud providers.

We got lucky: our customer shared his own case. This is a story with real figures and requests – about the situation with local hosting providers and the issues that came from cooperating with them, about how the customer’s entire data base was «accidentally» deleted during data tranfer, and about the importance of choosing the right provider. This case demonstrates that good marketing does not guarantee a good service. Let’s give the floor to the customer.

“Everything started when we, a large pharmacy franchise, decided that we needed a website and hosting for it. Sounds straightforward enough, but we had a big catalogue with a great amount of photos. Despite our expectations, the project only took off three providers later :). Later, we started renting servers and then we migrated to the cloud… But first things first.

A couple of words about us: “DS” pharmacy franchise, 111 pharmacies, 25 000 customers serviced daily, main area of operation is West Ukraine – Lvov, Volyn, Ternopol, Chernovtsy, Ivano-Frankovsk, Khmelnytsky and Zakarpattia Oblast.

IT Infrastructure for Pharmacy Chain

The sale of medication online is prohibited in Ukraine, which is why we created a separate list of products for online sale. This list included cosmetics, non-medicinal drugs (vitamins, for example), medical equipment, etc. The first task of the page was to inform customers: product description + photos, and the second – selling an assortment of goods appropriate for online sales.

The site was based on Bitrix and integrated with 1С. We enlisted the services of a Ukrainian provider to host the site. We never managed to upload the entire catalogue during the trial period. After that failure we thought that the problem lied with that particular provider. We tried another one and failed to upload the catalogue once more. We suspected that something was wrong with our software; however, that was not the case. We tested everything on our virtual server and saw that everything was working just fine.

We had to start searching for a hosting provider whose capacities could handle our vast catalogue. After 2 negative experiences with Ukrainian hosting, we decided to try the services of a European provider. We haven’t regretted that choice since.

With SIM-Networks, we uploaded the catalogue on the first try, without any delays. Ever since that moment, the hosting has worked well for 4 years.

Leasing other parts of our IT infrastructure

After we hosted the website, we wanted to secure our infrastructure, the ERP system and our data as a whole due to «certain actions» of the authorities (this was back in 2013). We decided to migrate our infrastructure to the cloud. Our servers could not handle the peak load and the capacities needed to be increased. In particular, we were concerned with the document flow upload (see below). Our first idea was to switch from SAS drives to SSD on our own servers, but after a number of calculations we realized that it would be more financially viable to lease the required capacities. With our rate of growth, we’d have to purchase expensive equipment every year. Naturally, this isn’t a viable strategy.

We surveyed the prices on the market and chose a domestic hosting provider due to the lower price. We took the capacities for a test, downloaded a copy of our system; at the time, everything seemed to function well enough. However, we were very disappointed once we launched the systems and started working. Throughout 2014, we had observed constant access failures leading to great delays in work.

Access failures with Ukrainian hosting. How was it?

It was «amazing». For example, infrastructure productivity would fall for 1-3 minutes and then quickly rise back up. This seemingly small delay caused a huge snowball of larger temporary delays and became the source of serious violations in our business processes.

sim-networks client review

In this specific situation the problem looked like this: the failures that lasted «only 3 minutes» created a huge queue to our data base, the 1С system broke through the block in these 3 minutes and reset all the documents in divisions and stores. Let’s just say we weren’t too happy with that outcome.

At the level of a specific employee, the process reminded us of a thriller film. For example, let us imagine that a manager downloads several waybills and the failure happens; the manager has no idea whether the transaction passed or not. Just like his colleagues, the manager now has to delete all downloaded transactions and manually enter the waybills. It was a boring, irritating and useless process that seemed like it lasted forever. Our monthly turnover was about UAH 50 000 000, and because of these failures the company could have lost up to 10%, i.e. about UAH 5 000 000 per month!

I’m not comfortable with generating bad press, which is why I won’t name this hosting provider.

Our peak load is approximately 700 documents downloaded as packages in the morning, from 9.30 to 11 a.m., and the same amount in the afternoon – from 3 to 5 p.m. That’s approximately 15000 documents daily. The documents were downloaded at once and and the same time and went through different processing – the checking of 1 waybill may be carried out by 20 (!) positions.

We should mention the specific seasonal fluctuations of such failures with our former Ukrainian provider: as a rule, they happened in the beginning and the end the month (around 10 days), the middle of the month (when everybody submits documents to the Pension fund) and the 20th day (tax document period)”.

Comments from SIM-Networks:

Such delays usually happen when the server is designed for, say, 5 clients: the provider places 10 customers there, naively hoping that they won’t use all of their capacities, which they have already paid for. We believe that it does not matter if the client uses the capacities they lease – as long as they keep paying, these capacities are theirs to do with as they please and theirs only.

The customer continues, «We were baffled that the provider did not foresee the possibility that clients may use all of their capacities at the same time. As a result, the resources that were guaranteed by the provider and that we ordered and paid for was not provided! There were no failures when our neighbors weren’t using their capacities.

We spent a lot of time proving to the provider that the issue was on their end – we started monitoring, wrote letters every time the failures happened. Only after a year of this nightmare did the provider agree to buy a SSD-rackmount in their data center, «special for us». It was still too early to be optimistic.

The migration of our data to this rackmount was performed hideously. The provider promised to transfer this data base within a day, on the weekends. When dinnertime on Sunday came, the restructuring of the new rackmount has not been finished yet, and we asked the provider to transfer the data back where it was. The provider insisted that the system was up told us that they deleted the old data base. The provider decided to delete the data, since they thought that the migration had already taken place and they didn’t have to store it anymore We were utterly shocked. In other words, they’d just destroyed our old data base before the new SSD-rackmount was even restructured, and we had to launch our entire retail network.

In view of such an emergency, we set up all systems from backups. The full restructuring of our «new» SSD-rackmount ended up taking an entire week. During this time, we had to work on our backup infrastructure, which was significantly slower. We don’t need to tell you what stopping for an entire week means for an online retailer.

We were ready to rent physical equipment from this hosting provider. They informed us that we’d need approximately USD 85 000 and offered us to buy this equipment by ourselves for installation at their data center. Naturally, we weren’t interested.

In search of a solution, we started tested various hosting providers – both domestic and European – and saw the difference.

It took us 16 minutes to download 100 waybills on our own servers. With 5 employees downloading 100 waybills, this took up to 2 hours and these 16 minutes turned into 40 minutes – 1 hour.

On the SSD-based hosting of our previous domestic provider, the same process took 9 minutes.

SIM-Networks - Fast!

With SSD-based hosting in a European data center, everything was downloaded within around 4 minutes. The main task we were trying to solve was the liquidation of queues to the data base, and the capacities of SIM-Networks helped us achieve that.

After accumulating unsuccessful, but extremely useful experience, we decided to rent physical infrastructure instead of virtual capacities from a European provider. We first transferred essential 1C services to the new infrastructure. After the initial success, we rented 2 additional reserve servers and transferred our cluster, which included a file server, Microsoft Exchange, etc. As a whole, we rented several servers and a rackmount in the data center.

On the financial viability of IT infrastructure rental

When we were thinking of switching to SSD drives, our servers had already been in operation for 3 years, and the warranty from the manufacturer was coming to an end (next business day). We had to sell it as soon as possible. This warranty required about UAH 300 000 (approx. USD 25 000 at that time) annually plus expenses for electrical power.

Calculations showed that we decided to purchase the equipment ourselves, we’d break even on capital expenses in 3,5 years. Taking continuous development into account, the amount of expenses on our infrastructure could increase drastically.

Consequently, renting IT capacities was relatively affordable and rid us of:

  • warranty prolongation and purchase;
  • the need to increase of physical capacities (equipment upgrade);
  • expenses for electrical power.
Public Cloud

Public cloud

Learn more about the highly available public SIM-Cloud

Learn more

We are going to the cloud!

We want to expand again, since the capacities we have at the moment aren’t enough. We are seriously considering the cloud as an alternative. We don’t have any concerns regarding cloud migration; experience taught us that matter where you will deploy your infrastructure doesn’t matter as much as who you’ll be renting it from. Choosing a reliable service provider is the key!

Andrei Semeniuk, IT-Director, D.S. Pharmacies network (Market Universal LTD)

sim-networks-client-review-ds-timeline-eng

Did you like the article?

Cookie consent

By clicking «I agree», you consent to our website's use of cookies to give you the most relevant experience by remembering your preferences and repeat visits. However, you may visit «Manage сookies» to provide controlled consent. Learn more

Cookies settings

functional

Necessary cookies are crucial for the basic functions of the website and the website will not work in its intended way without them.

Analytics

Analytical cookies are used to understand how visitors interact with the website.

Advertisement

Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns.