Alternatives for Integrating Magento with Third-party Applications

This article is a case study for integrating Magento with other systems when web services are not suitable, and we will go through some real-life examples.

By default, Magento implements two types of web services: SOAP and REST. This does not require that you know how Magento is built, how it works internally. You can perform almost any operation that Magento can do by default (create orders, add customers, get list of products, etc.) just by consuming web services.

Some examples of usage:

• you need to get the list of orders in your ERP software;
• your courier assigns AWB to orders and marks the orders in Magento when they are shipped;
• you want to show a few products on your blog;
• etc.

However, this approach is not always suitable, because things can get messy. You may need an intermediary software to achieve the results you want.

Middleware Applications

Note: in this case the term “Middleware” does not refer to the design pattern used in microframeworks (although this could help in the implementation).It refers to a piece of software that stands between two systems helping them communicate.

Case study #1: Multi Store Stock Middleware

I once worked on a project which had, as part of its multichannel selling strategy, integrated a chain store and an online shop. Nothing fancy, the usual business scheme for this context. However, the challenge arose when we received the request to update the stock of products (which is common for all selling points) in real-time on Magento and consequently display it on the storefront.

The beauty was that the chain store did not have a dedicated warehouse. They had approximately 100 physical stores. Each of these stores shared the stock with the online store (half / half). Every time products were sold in the physical store, we had to update the stock in Magento. This could have turned messy if all 100 stores started to update the stock at the same time, with the Magento API.

So we built a Middleware which centralizes the data and updates the stock in Magento (SOAP) in a transaction safe manner. Below you can see a simplified version of the Middleware’s flow diagram.

Some notes about the Middleware we created in this case:

• it can be accessed by web services (SOAP, REST, JSONRPC);
• it had access to other systems also by web services;
• it had its own database;
• it is the owner of the stock – that’s why when changes happened, the Middleware needed to be updated.

When you should use this approach:

• when there are multiple data sources, and data need to be centralized before storing it in Magento;
• when logic can be set apart from Magento;
• when the load would be high on Magento, in which case the Middleware can work on a separate environment without affecting the load on the Magento server.

When you should NOT use this approach:

• when you can solve the problem just by using Magento web services (there is no concurrency between client sides).

So, how did we build this application?

I used Zend Framework 2 for implementing this middleware along with Doctrine DBAL. I implemented a SOAP server in Zend, which is similar to Magento’s SOAP implementation. It is WSI compliant, and you need to have a valid session ID to access any of the methods. You can get the session ID by accessing the “login” method of the SOAP server, where you need to provide a valid user and password.

Whenever the stores (ERP) require a stock update, they access this SOAP interface. The middleware receives an SKU of a product along with a number indicating how many items should be added to the stock. This number can be positive ((for adding new products to the store) or negative (for sold products in the store). The ERP has significant limitations and doesn’t offer much flexibility for updating products. At first, I wanted to put all incoming requests in a queue, update each of them and notify the ERP. This unfortunately was not possible, I had to send a success or failure message on each request. So as a response, I return a success or failure message on every call, depending on the update status. If the response was a failure, the ERP tries the update later.

On the Middleware side, I store the stock in a MySql server, using the INNODB storage engine for the stock database table. This is essential, because it supports transactions and row level locking. So whenever a request comes, the row of the SKU is locked, so no other request can update it, and this lock is kept until the stock is successfully updated in Middleware and Magento. Since Magento only updates the stock and enables caching for WSDL, this process works quite quickly, within a second.

In case two requests are trying to update the same SKU product, the first one locks the row and the second, if it cannot update the stock, it waits a few microseconds, and then retries the update. It keeps doing this for a limited period and if it still cannot update the stock, it returns an error message to the ERP system. This kind of fails are quite rare, they occur sometimes during mass updates.

Obviously, this application is more complex, but I will not get into further details, because this is enough to understand the big picture.

Case study #2: Commissioning system

The second time I put this same decoupling principle into practice was for a commissioning application for an MLM (multi-level marketing) business. Through extensive effort and creative solutions, we customized Magento to accommodate the exceedingly intricate MLM business model. This model encompassed challenging formulas and rules for assigning commissions and bonuses to users across all levels of the MLM hierarchy tree. Because these calculations were time-consuming, but could be completely separated from the online shop, we decided to build a separate application for this purpose. The purpose was to fetch data from Magento, make the calculations and send results back to Magento. This application was able to run on a separate environment, with optimized settings and without affecting Magento’s flows and logic in any negative way.

This is how the high level flow diagram looked like:

When you should use this approach:

• when the business logic is complicated;
• when the logic is not directly related to Magento;
• when the load would be high on Magento – the Middleware could work on a separate environment without affecting the load on the Magento server.

When you should NOT use this approach:

• for small tasks.

Some Technical details Implementing Middleware (in PHP)

• if you are using PHP to implement the Middleware, it would be ideal to use a micro framework (Zend Expressive, Silex, Slim) because you don’t always need the complexity of an MVC framework, and they have well implemented routing systems that will make your webservice implementation easier;
• in case the load is very high, and many concurrent clients try to access the Middleware at the same time. You can use a queue system to intercept and distribute the requests (Ex.: Rabbit MQ, Active MQ);
• it’s highly recommended to use a RDBMS that support transactions (MySql, PostgreSql, etc.);
• the Middlewares I worked on were written in PHP and used Doctrine for data manipulation. At first, the implementation was based on Doctrine’s ORM. Soon we gave it up, due to performance issues (it added some extra time in the execution). And we relied only on Doctrine Database Abstraction Layer – which worked perfectly fine.

Using SQL views

Besides Middleware there are other alternatives for integrating Magento with third parties. There are cases when Magento web services can be quite slow, especially when we want to transfer large amounts of data.

Case study #3: Reporting

For example, we had to create some Business Intelligence reports for a project. The software that we were using for data centralization did not support the complex SOAP implementation that Magento has. On top of that, performance was also an issue. So, we decided to use MySql views. Mind you, this requires a bit of understanding of how Magneto stores data, and some advanced query writing skills. But done properly, views can make miracles in terms of speed.

Example of fetching customer related information from attributes without hardcoding the attribute IDs:

You should use this approach:

• just for reading data (set MySql user permissions accordingly);
• when time is crucial;
• when you have to transfer large amounts of data.

You should NOT use this approach:

• for inserting or updating data;
• if you don’t know how Magento works internally.

Conclusion

Sometimes you have to add another layer or remove the layers you don’t need, when the tools Magento offers do not meet your needs. When you have a business logic that can be separated from Magento, it’s a good idea to do so. Utilizing web services will simplify your approach in this aspect.

Article written by Csaba Varga

This article was originally published here.