While applying Odoo for large corporate entities, we crossed paths with HULFT. This technology definitely brings added value, performance and reliability when it comes to exchanging data between Odoo and other systems which do not have real time API features.
Although HULFT is NOT an API system, it allows a highly efficient and fast data exchange and becomes a powerful ally when you put Odoo at the heart of transactional systems requiring a “close to real-time” data update.
Denis Guillot, CTO of Port Cities, provides more details on his experience with the powerful managed file transfer (MFT) middleware in this informative analysis. Let’s find out more about HULFT in comparison with other standard file transfer protocols (SFTP).
A little bit about HULFT
HULFT 8 is next-generation MFT middleware used by leading enterprises for sending mission-critical data reliably, securely, and efficiently. Robust and powerful enough to be used as the foundation of your business infrastructure, HULFT can handle character code conversions, connecting with different operating environments, and dealing with a range of file and code system types. By using industry-standard TCP / IP protocol security, HULFT delivers your data faster and won't over-burden your network.
As would be expected, HULFT is compatible with UNIX, Linux, and Windows. It also supports mainframe operating systems such as z/OS, MSP, and VOS. HULFT even works with mid-sized systems running i5/OS.
With HULFT, seamless data integration between a wide variety of business applications is possible. Transfer files between your enterprises' different locations, or with external organizations. HULFT is becoming the standard for managed file transfer (MFT).
HULFT vs. standard (S)FTP
In this section, we compare bi-directional file exchanges between 2 systems with SFTP and HULFT.
| HULFT | SFTP |
| Provides a triple-combined solution per instance giving us: ● Live origin and destination volumes redundancy / fail-over ==> HULFT was capable of mirroring the files in multiple locations and guarantee exchanges bilaterally, even if the server of the client failed. ● HULFT is a triple layered communication system capable of detecting origin failure or destination failure and differing, postponing, resuming a queue depending on the health of instances on the sending or the receiving end. |
● With standard FTP, there is no failure detection and decent auto-resume feature, and even less on a list of files which grows every 5 seconds. |
| HULFT can guarantee file transport while FTP only promises it. | With SFTP, you need to install 2 FTP servers and 2 FTP client services inside two instances and shoot files "blindly" in both directions, hoping that the two sending FTP clients and the two receiving FTP servers are online, available and working. |
| HULFT is capable of creating a "to-do" list of files to be transferred between different instances and execute it. | With SFTP, we rely on 4 independent individual processes with no coordination... We just push and "hope" it will be received. |
| HULFT is capable of delivering a full file integrity check and validation on the receiving ends. This means an error-free transfer. In case of error, corruption or segmentation, transfer shall be re-initialized and executed properly. | With SFTP, again we push and hope everything goes well. We assume the file is correct. With a new file generated every 5 seconds, we are looking at a world of problems. |
| HULFT has a high performance file manipulation service which allows file manipulations to be executed at a very high pace. Coupled with ObjectiveFS, we managed during POC to trigger live transport of 500 CSV files of 20Kb within 1 second between 2 different HULFT containers. | With FTP, with the same conditions and using a local internal network in the same DC, we needed 3 seconds latency, only to push the first file of 20Kb, just because the FTP services require the standard authentication time. After authentication, we managed to push 1 to 5 files per second, depending on the FTP server buffering and threading. When we decided to exchange transaction CSV files from third-party systems to Odoo at a high pace, SFTP became a big liability. |
| HULFT operates a real-time "in-transit" compression of the files it manages. For a CSV of 20kb, the HULFT service crushed the payload down to 1.64 Kb during our POC. This is not only explaining the speed.... it is also very important when the files you send from Odoo are going to get bigger and more numerous. | While HULFT has no problem dividing this by almost 20x, SFTP does not and this implies that time in transit is going to increase very rapidly, making a quick data refresh impossible. |
| HULFT comes with HULFT SCRIPT, which allows to manipulate, rename, move and process the files and the file queue on both ends, before they are sent and after they have been received. This includes generation of date:time stamped file names and the relocation to archive of files which has been processed. | FTP technology does not allow this file life-cycle processing and this implies we need to write more scripts either at the OS level or the Odoo level to manipulate these files and move them out of the way once they have been sent or received and processed. |
| Odoo Crontab’s scheduled tasks have a minimal execution gap of 1 minute. What happens when you need to process faster than that? |
| HULFT can manage all file exchanges, no matter the load. | With FTP, we shall need to take up the following challenges: ● declare allowed users on both ends. ● determine allowed simultaneous connections limits. ● setup a connection time-out limit (for when the queue gets stuck over one transfer and everything stops) ● make sure at all times that the connection time-out limit is not too low... otherwise, it would cut the transfers when there are more files and they are bigger and require more time. |
| HULFT comes with an out-of-the-box security protocol resting on a one port multiplexed service. | FTP and SFTP on the other hand are a lot more basic. We will need to manage bilateral server and client credentials on both sides and pay more attention to the security aspect since FTP / SFTP are the number one target of hackers and bots. |
Fine with HULFT, but what about the Odoo interaction?
HULFT SCRIPT allows usage of:
Script triggers: Monitors HULFT’s transfer log files, and executes processing if a file ID, etc., meets certain conditions.
File triggers: Executes processing when files to be monitored are newly created, updated, or deleted.
Scheduler triggers: Executes processing according to a set schedule.
Starting from there, triggers can be directly used, related with or manipulated by:
Odoo scheduled tasks.
ORM events.
Triggered actions from ORM functions/widgets.
HULFT & HULFT SCRIPT advantages in a nutshell
There is a very good reason WHY HULFT was so successful at delivering file transfer and exchange solutions for corporate banking, insurance and retail industry:
Reliability
Simplicity
Security
Performance
As you can now understand, standard SFTP-based solutions offer none of the above 4 qualities.
Meanwhile, HULFT SCRIPT allows all the necessary operations to be automated and integrated within the Odoo ecosystem.
The field of application becomes very wide and allows to automate any “close to real-time” data import / export / update process between Odoo and previous generation transactional systems which do not offer API features but require constant communication.
Are you a large corporation who needs assistance in connecting Odoo with other systems that require large exchanges of data? Message us and our consultants are happy to help.