Guest post by Transmetrics – a Big Data predictive analytics solution that is tackling capacity utilization problems in cargo transport by predicting and optimizing capacity utilization. Transmetrics works with some of the leading logistics companies in the world, including three global Fortune 500 companies.

As a means of cutting transportation costs from third party freight companies, eCommerce giant Amazon is rolling out its own fleets of air, road and sea vehicles. Amazon already boasts a 40 strong fleet of Boeing 767 aircraft, has registered for a license for sea freight and has had a patent approved for plans for a futuristic, high speed subterranean urban delivery system. The arrival of well funded, global tech players should set off warning alarms for traditional logistics companies, however, the real disruptive force isn’t from underground pipes, ships or planes. It is from effective use of data.

“Based on volume, scale and buying power, Amazon will command more attractive pricing than other freight forwarders, enabling them to secure capacity at a lower cost and ensure profitability as they fill that space more easily than competitors.”

Amazon uses some of the most advanced analytics and technology available and is constantly using data and experimenting to improve its delivery model. On the flipside, the traditional logistics industry is being held back by legacy mentalities and data sharing policies. However, to survive in the new environment, companies need to realize that they are no longer competing amongst each other, but with data-driven global giants.

A recent Logistics Management study argues that if logistics companies don’t adapt, “Based on volume, scale and buying power, Amazon will command more attractive pricing than other freight forwarders, enabling them to secure capacity at a lower cost and ensure profitability as they fill that space more easily than competitors.

”Here are the keys for logistics companies to unleash the full potential of their data by using artificial intelligence and machine learning and smartly exchange their data with other logistics players, in order to stay competitive against companies like Amazon:

1. Recognize the need for a shift in mentalities

The Transport & Logistics sector is a highly competitive and crowded industry in which companies operate on very thin profit margins. Most leading logistics companies can’t compete much on the products they are offering, but instead on price and speed. As such, companies have traditionally considered their fleet data to be a private treasure chest to be guarded at all costs from the prying eyes of competitors, so as not to risk offering them a competitive advantage.

However, logistics companies need to realize that by hoarding their data, they are actually putting themselves at a disadvantage.

Since venturing into the world of logistics, Amazon has been able to create data from the start to the end of the product journey. This ‘bird’s eye view’ of data across the whole supply chain from manufacture to delivery poses a huge challenge to the more fragmented system used by logistics companies. And as a recent Logistics Management report puts it, only the most analytically fit will survive.

This visual shows the logistics process in two parallel flows, labeled A and B, depicting the stages involved in shipping and delivering goods. Flow A (Top Row): Flow A illustrates the sequence from initial pickup to final customer service in a logistics network: First Mile Pick-Up (Kaban/People): The process begins with the collection of goods, shown with a truck loaded with packages. Warehouse (Management/Consultant): After pickup, goods are sent to a warehouse for storage and management. Middle Line Haul: The packages are transported in bulk through a middle line haul, which involves moving goods over longer distances. Last Mile Delivery: This stage represents the final delivery to the destination, where goods are transferred to delivery teams for distribution to customers. Customer Service: The flow concludes with customer service, represented by a headset icon with "7x24h" to indicate around-the-clock support. Flow B (Bottom Row): Flow B illustrates the international shipping stages, starting from the point of origin to the final inbound service: Departure Port Service: Goods are prepared and loaded at the departure port, ready for international shipment. Air/Ocean Freight Service: The shipment is transported via air or ocean freight to the destination country. Destination Port Service: Once the shipment arrives, it is unloaded and processed at the destination port. Line Haul: Goods are moved overland in bulk from the destination port to the next logistics center. Global FC (Fulfillment Center) Inbound Service: Finally, goods reach the Global Fulfillment Center, where they are processed for the last stages of distribution. These two flows (A and B) work in parallel to demonstrate both domestic and international logistics stages, covering each phase from pickup to final customer service. The icons and arrows illustrate the sequential steps, emphasizing the logistical chain's structure and complexity.

Whereas all Amazon factory workers, packers, drivers, and delivery workers log information into the same data systems offering a more fluid data trail from start to finish, individual logistics companies generally only manage particular steps of the journey and don’t tend to share information with those manning the other parts of the system. This limits the actual learnings that companies can take from their data, and thus limits how actionable the insights are which the data can offer.

2. Push for better data

Another major roadblock for logistics and transportation data management is the fact that companies are not taking adequate steps to ‘clean up’ their data.

As mentioned before, the data trail for companies like Amazon can follow a package or shipment from beginning to end on the same system. Although more focus has been placed on the importance of effective data management in recent years, many leading L&T companies fail to enforce unified data standards when recording data, which leaves gaps and makes it very difficult to use or potentially share.

“Although more focus has been placed on the importance of effective data management in recent years, many leading L&T companies fail to enforce unified data standards when recording data, which leaves gaps and makes it very difficult to use or potentially share”

For example, when logging data about shipping containers, or truckloads, there is still no standardized measure. Even within the same organizations, members at different parts of the process are not recording shipments in a uniform manner. Some may register volume, others loading meters, and others weight, and many companies are failing to even register the size and capacity of different types of vehicles. This disorganized information is all but useless on a big data system, unless it is calculated and logged in the same measurement, which makes fleet optimization to improve efficiency all but impossible as companies are unaware as to how full their truck loads are. Advancement is also being held back by continued use of legacy systems. Many companies still require employees to note and enter data manually and accept clients’ estimations of the weight and volume of shipments, which inevitably leads to human error.

To move past this, L&T companies will need to automate and harness the power of AI systems to streamline the process moving forward, but also to ‘clean up’ existing data. This means investing in more technology, and in more data talent as well. Leading eCommerce companies like Amazon or Alibaba have the resources to invest in large data teams, but to date, many logistics companies have continued treating analytics as a side project, not their main focus.

However, hiring data analysts, developing native predictive optimization tools, and cleaning up data is a slow process, and time is not a luxury being offered by Amazon. A quicker option would be for logistics companies to work with external providers who specialize in AI, machine learning and logistics, to move the needle, quicker.

3. Use data to fill in the gaps

Data only works when it covers the whole story and at the moment there are too many gaps. While a logistics company may be able to gather data about a specific shipment from A to B, they are left in the dark about the legs preceding and following their involvement. Until logistics companies work out a means to share selected parts of their data with other players, no one is going to have a solid overview… aside from Amazon.

Logistics companies need to cooperate in building a data-driven process based around transparency, which would allow them to reach a “single version of the truth” and allow them to stay competitive.

At current, the system to track shipments and freight across Europe is surprisingly underdeveloped. In a recent Financial Times article, it is highlighted that European Security officials have shown concern at the fact “There is currently no comprehensive system to track shipments and cargos through EU ports and along its approximately 70,000km of coastline.”

Despite a recent report suggesting as much as half of the freight in the EU is transported by sea, the level of data sharing between ports and logistics companies is very limited. Some of the biggest European ports, like Rotterdam, Hamburg and Antwerp all have their own smart data initiatives which log and track shipments passing through, but despite calls for a more cohesive system, they are still not exchanging data. A recent Deloitte study argues that seaports need to catch up with leading transport and logistics companies, however, it would be more appropriate to say both sectors need to catch up with Amazon.

4. Looking to the future

While companies are unlikely to willingly hand over their precious data in its entirety to their competitors, they may feel more comfortable sharing data with a third party. This could be a non-profit organization governed by the EU or another governing body, which would be trusted to use the data for research and to create a usable overview from many sources of data without specifying the original source company.

In his study “Toward a Physical Internet: meeting the global logistics sustainability grand challenge,” Benoit Montreuil proposes the creation of the ‘physical internet’ based around collaboration between academia, industry, and government across localities, countries, and continents. The physical internet is a global logistics system that transports products in standard-sized, modular containers as seamlessly as the internet moves digital information.

“Companies may feel more comfortable sharing data with a third party. This could be a non-profit organization governed by the EU or another governing body, which would be trusted to use the data for research and to create a usable overview from many sources of data without specifying the original source company.”

Another option would be to collectively store and share select data on a decentralized ecosystem using blockchain technology, which would allow individual companies to choose what information is shared with whom. This ecosystem could be made available only to agreed participants, following agreed rules, and sharing data in the agreed format about agreed themes. Companies wouldn’t need to share customer sensitive data, and the data could be perfectly anonymised while still offering better visibility along the whole process.

As part of the blockchain, logistics companies would be able to create ‘smart contracts’ with other members of their data ecosystem which govern when and how data was made available. These smart contracts could also open the door to other ‘share economy’ functions, such as shared fleets of autonomous delivery vehicles which could be shared by different logistics companies in urban areas to cut costs all around.

Whether it be in sharing data, or sharing ‘on demand’ vehicles for last mile delivery, logistics companies need to start building bridges rather than walls. If the EU and individual companies don’t find a way to cut costs and make data-driven decisions soon, it may well be the nail in the coffin of logistics as we know it today. Traditional companies can only rest on their laurels for so long before they are forced to adapt or disappear. The clock is ticking, and with every moment that traditional companies hesitate, Amazon is creating, storing and analyzing more data, and taking one step ahead of the competition.