admin, Author at SHIP IP LTD

LONDON — Humans aren’t the only ones to have had their travel plans ruined by the coronavirus. A robot-powered boat that was due to cross the Atlantic this month has been forced to delay its voyage until next April after the virus caused complications in its development.

The autonomous 15-meter trimaran has been built to push the boundaries of autonomous shipping while gathering scientific data on the ocean. The Mayflower Autonomous Research Ship (MAS for short) is being led by marine research organization ProMare, while IBM is the main technology partner.

The solar-powered vessel is set to start trials off the south coast of England in the coming weeks and it will be officially unveiled on Sep. 16, the 400^th anniversary of the Mayflower departure in 1620. After that, it will go on several voyages and missions over the next six months ahead of a transatlantic voyage in April 2021.

During that transatlantic crossing, the ultramodern ship will broadly retrace the Mayflower’s original route from Plymouth to Cape Cod’s Provincetown.

A computer-generated image of the new Mayflower vessel.

University of Birmingham’s Human Interface Technologies Team (HIT)

Andy Stanford-Clark, chief technology officer for IBM U.K. and Ireland, who’s leading the science on the ship, told CNBC that the vessel relies on an onboard AI Captain which uses computer vision, automation software and Watson technology — IBM’s most notable AI platform.

The ship’s operators tell the Mayflower where they want it to go and then it will figure out how to get there itself, considering the weather, ocean currents, collision regulations and other variables. The Mayflower can also react to ocean traffic in real time using a combination of radar, cameras, and the Automated Identification System (AIS), which transmits information such as the Mayflower’s latitude and longitude to other boats.

While AI does not control every aspect of the Mayflower, it does play a significant role in the ship’s operations.

Onboard experiments

Inside the boat, instead of there being crew, living quarters, a bathroom, a galley, beds and so on, there are science experiments.

Various organizations have pitched experiments that they’d like the Mayflower to carry out while at sea.

One of them is a water analysis experiment that samples the seawater every few hours and stores it in roughly 100 bottles that are capped and kept in a crate onboard ready for human inspection back on land. “Because we’ll know exactly where in the ocean we took the sample, we can say at this point, the salinity was this, the algal bloom quantity was this, the pH was this, and the oxygen levels were this,” said Stanford-Clark.

The Mayflower ship's main hull being transported to a wharf in Plymouth for final assembly.

The ship’s main hull being transported to a wharf in Plymouth for final assembly.

IBM

Scientists at the U.K.’s Plymouth University will analyze the same samples to determine microplastic levels at various points in the ocean.

IBM claims it has also developed a system that can identify whales and the pod they’re from based on their song, which is picked up by an onboard hydrophone. “We’re training an AI to listen for whale songs,” said Stanford-Clark. The same hydrophone is used to listen out for faults on the Mayflower.

There’s another project that will involve analyzing the shape of the Earth. “We’re going to use very accurate GPS to measure exactly the level of the ocean, and then subtract the tides and the weather and the wind and use that to get a unified model,” said Stanford-Clark.

Journey time

The original 30-meter Mayflower took 66 days to carry pilgrims from the U.K. to what is now the U.S. But the new one will take two to three weeks, depending on weather.

Director of the Mayflower project, Brett Phaneuf, told CNBC that the new Mayflower is packed full of technology the original pilgrims wouldn’t have been able to comprehend. But it’s important not to get carried away. “It’s not a Terminator,” he said. “It’s not going to take over the world and it’s not going to chase all our ships off the ocean. It’s a democratizing technology that will help us collect vast amounts of unfamiliar data about the ocean.”

“The interesting thing about Mayflower is there’s nobody to get tired or bored or lonely or injured,” said Phaneuf, pointing out that the vessel can go as fast or as slow as it likes for as long as it wants. “If something is interesting, or new, it can be diverted for next to no money.”

There’s every chance that the ship will encounter problems on the transatlantic trip but Phaneuf believes it’s robust enough to make it. He said it will take something idiotic and mundane to stop it, such as the rudder getting jammed by a log.

Covid-19 disruptions on manned research vessels

The new Mayflower could signify a change in the way ocean research is carried out in the future.

Coronavirus disruptions on manned research vessels are leading to a reduction in data about the weather and climate change, according to UNESCO’s Intergovernmental Oceanographic Commission.

Reduced air travel means there’s a dearth of weather data from the skies as well.

All these disruptions could result in less accurate weather forecasts and climate models, the Commission has said.

The future of autonomous shipping

In many ways, IBM is testing the water for the future of autonomous shipping with the Mayflower.

Allied Market Research thinks the autonomous shipping market could be a $135 billion industry by 2030 and IBM is weighing up where and how its technology can be used.

Stanford-Clark believes that AI captains could be used to “look over the shoulder” of a human captain.

“That same technology that we’re putting in Mayflower will also be able to operate in a guardian angel sort of mode,” he said, adding that there’s a huge amount of interest in this technology from big shipping companies.

He believes there’s a “big opportunity” for container ships to become autonomous in the future, adding that it would help to keep supply chains open during pandemics.

Autonomous cars get all the attention but autonomous shipping is potentially even more useful.

“Everyone wants to see what we’re doing with Mayflower first but people are lining up to have these kind of conversations,” said Stanford-Clark.

Source: cnbc

The British Ports Association (BPA) has opened a call for evidence from the wider maritime community on what ports should consider to prepare for receiving autonomous ships in the future.

This is part of its new initiative looking at the implications of autonomous shipping for UK ports. The call for evidence will feed into the BPA’s own MASS analysis as well as its discussion with industry partners through bodies such as the industry umbrella group Maritime UK. As part of the initiative, the BPA is also creating a new Autonomous Shipping in Ports Network.

Richard Ballantyne, chief executive of the BPA, said: “The prospect of seeing autonomous ships in UK ports is definitely on the horizon but there will be much to do to prepare ourselves. This includes the consideration for port and marine operations, regulatory frameworks, infrastructure and receptions facilities, land to vessel communications and vessel safety.

“Although we are at the early stages, a lot of work is taking place across the maritime community. Many UK ports are now starting to ask what they should be considering as they develop their ports and recruit the next generation.

“This will be an ongoing initiative for the BPA but initially we are inviting evidence from a cross section of maritime sector organisations. This will be in relation to issues that they see specific in to ports and harbours moving forward into autonomous maritime operations. It will also help us to support and participate in various government initiatives in the UK such as Maritime 2050, the work of the Maritime Skills Commission and other innovation, sustainability and infrastructure projects.”

The BPA’s new Network will be open to all its port members, but it will also be drawing on external expertise from specialists working in the marine and MASS sectors.

The BPA will accept submissions to the call for evidence until 12 October.

Source: portstrategy

InterManager – the international trade association for ship and crew managers – has today published an open letter addressed to Mr Kitack Lim, the Secretary-General of the International Maritime Organization, on behalf of its members and associates.

The letter urges Mr Lim to offer more support to the global maritime industry with regards to the Singaporean repatriation issue.

Several major seaports throughout Asia have tightened crew change restrictions and stepped up screening of seafarers in recent weeks as the coronavirus stages a global resurgence.

The Maritime and Port Authority of Singapore recently issued correspondence stating that “the MPA will be giving priority to crew change applications from Singapore-registered ships and applications for sign-off crew only, without signing-on new crew,” a release which has prompted InterManager to issue an open plea to the IMO’s Secretary-General.

Speaking of the situation, InterManager’s Secretary-General Capt. Kuba Szymanski said: “If the same rules issued by the MPA of Singapore were applied to all Flag States, it would pose a very dangerous narrative, as ships of other Flag States would not be allowed to perform crew changes anywhere in the world.

“We – ship managers – would be unable to change off-signing crew because they would immediately be in breach of safe manning regulations.

“We completely understand the concerns that Singapore has, and we support their efforts to look after their citizens. However, no ships means no supplies, so a collaboration between the shipping sector and local administration is of paramount importance, and needs to be a two-way street.”

The measures implemented by the MPA of Singapore follows a set of best practices issued on 24^th July in light of a resurgence of coronavirus cases, which included isolation periods for on-boarding and off-boarding crews, as well as virus testing.

InterManager believes that the restrictions imposed on non-Singapore-registered vessels creates a myriad of issues to the safety of seafarers, and presents a significant breach of seafarers’ humanitarian rights.

“Their approach is everything but human-centric,” said Capt. Szymanski.

Capt. Szymanski said that officials in Singapore have less than concrete action to help crew repatriation in these troubling times. Recommendations suggested by the MPA are not practical, and InterManager is encouraging anyone who did manage to carry out crew changes in Singapore to get in touch in order to keep a good record of the scale of the problem.

“Presently, our records show zero crew changes for non-Singapore ships, for both on and off-signing crew changes,” said Capt. Szymanski.

Source: intermanager

There’s hardly anyone buying new ships, with orders plunging to a 20-year low due to a potent combination of uncertainty over environmental regulations, the economic fallout from the coronavirus pandemic and a lack of financing.

“The IMO has brought in significant, ambitious and important targets around emissions,” said Clarksons Research’s managing director Stephen Gordon. It remains unclear the exact policies and regulations that might be introduced and what technology will be adopted, he said. Ships are long-term investments, and buyers run the risk that their vessels will become obsolete.

The global shipping industry is in the midst of one of its biggest changes in a generation after stricter environmental rules kicked in at the start of the year. Ship owners face paying more for cleaner fuel, retrofitting ships with pollution-reducing scrubbers or even ordering new vessels. Compounding the uncertainty has been Covid-19, which has upended supply chains and stalled trade flows.

“Covid-19 has become the most immediate issue,” Gordon said. While challenges due to lockdown measures are easing, “the economic uncertainty, disruption to trade, and volatility in freight rates” caused by the virus are driving orders lower.

Demand growth for containers is expected fall this year due to Covid-19, according to A.P. Moller-Maersk A/S, which predicts a return to 2019 volumes in the early part of 2021. The world’s largest container line ordered only eight vessels in the second quarter, putting its orderbook-to-fleet ratio at 9.4%. Globally, the rate is about 8%, meaning orders for new ships are at a two-decade low, according to Clarksons’ Gordon.

“The virus is a further hit to demand that’s already barely even there,” said Rahul Kapoor, head of commodity analytics & research, maritime & trade at IHS Markit. “With the pandemic’s hit to economic activity and supply chains, ordering new ships is now the lowest priority for companies. They’re concentrating on just trying to maintain profit margins.” The virus has also delayed the completion of shipbuilding projects, he said.

Finance Crunch

Shipowners are also lacking the finances to make purchases, according to Ralph Leszczynski, head of research at shipbroker Banchero Costa & Co.

“Most shipping markets are coming from a relatively poor decade, 2009 to 2019, in terms of earnings so most shipowners do not have that much cash in their pockets,” he said. “External finance is also in short supply as banks are now largely steering clear off shipping after the defaults they suffered after 2008.”

Still, fewer orders and slower fleet growth will likely bolster shipping rates. Lines are likely to continue to keep capacity in check into 2021 to minimize the impact from slowing global trade, said IHS Markit’s Kapoor.

The cost of moving goods by ship from Hong Kong to Los Angeles surged

That’s already translating to increasing costs for transporting goods by ocean liner, with one benchmark of trans-Pacific container rates more than doubling since late-May to a record. Bulk-carrier costs have also rebounded from a four-year low. Maersk, which idled about 20% of its capacity in April before gradually reinstating it in subsequent months, saw earnings beat estimates in part due to improved freight rates.

Big Ships, Small Ships

The offshore sector has seen demand for oil rigs and supply vessels hammered as energy prices remain low and there’s little interest in new exploration investments, said Leszczynski. It’s a “gamble” to purchase an oil tanker amid uncertainty over demand, he said. While oil consumption has grown in the past two decades, climate change mitigation efforts are spurring market expectations that appetite will decline.

While the coronavirus adds to short-term uncertainty, there’s a better medium-to-long term outlook. Qatar signed a deal in June worth around $19 billion with South Korean shipbuilders for more than 100 liquefied natural gas vessels, Maersk expects a progressive recovery in volumes and port operator DP World said it’s positive on fundamentals.

The shipbuilding sector is set to remain subdued for the next few years, with a revival possible eight to 10 years from now, said Leszczynski. Vessels built during the boom years of between 2007 and 2010 will require replacement, as most have a lifespan of about 20 to 25 years, he said.

— With assistance by Kyunghee Park
Source: bloomberg

To allow for the “speedy and safe travels” of seafarers during the coronavirus disease (Covid-19) pandemic, the Maritime Industry Authority (MARINA) said it has communicated the country’s guidelines for the establishment of a “green lane” for seafarers to the International Maritime Organization (IMO).

In a statement on Tuesday, the MARINA said the guidelines, made in response to the IMO’s recommendations for safe ship crew changes during the pandemic, would allow for the safe and swift disembarkation and crew change of seafarers in the country.

“These guidelines were formulated through an inter-agency effort in order to expedite the travels of seafarers involved in deployment, disembarkation, and crew changes by establishing a Philippines Green Lane,” the MARINA said.

The Department of Foreign Affairs and other government agencies in July signed the Philippine Green Lane Joint Circular that would facilitate the creation of controlled travel lanes to stimulate the country’s economy through the safe and efficient movement of seafarers.

In a message, Luisito “Lui” delos Santos, officer-in-charge at MARINA’s Management Information Systems Service, said the IMO will issue a circular to its member states in response to the Philippines’ new policy on the travels of seafarers and crew change.

“This will serve as guidance for seafarer-foreigners joining their ships docked in the Philippines, disembarking from a foreign ship docked in the Philippines to an international gateway for purposes of repatriation, and foreigners transiting in a Philippine international gateway,” delos Santos said.

He added that other member states of the IMO are also encouraged to communicate to the IMO their policies and practices relating to crew change and the movement of seafarers.

MARINA said it has issued Advisory 2020-60 and 2020-61 that both highlight various crew change protocols from other IMO member states for the use of licensed manning agencies, ship owners, ship operators, and other concerned entities in the Philippines’ maritime sector.

On August 22, MARINA launched the country’s third crew change hub at the Subic Bay Freeport Zone—after the Port of Manila and Port Capinpin in Orion, Bataan.

Late in July, MARINA released new protocols that limit crew change activities to crew change hubs or ports that have been adapted for health and safety procedures such as testing for Covid-19 upon disembarkation of seafarers and crew changes, among others.
Source: PNA

Further digitalisation in ports is increasing their vulnerability to hackers and cyber attacks. As more technology is linked to the internet, the frequency of these threats and chances of a successful breach increases.

Cyber security needs to be improved in ports before internet of things (IoT) is introduced into port infrastructure. With more automation in ports, some of these networks are overlooked by IT teams and could be vulnerable to hackers, said University of Plymouth, Faculty of Science and Engineering lecturer in cyber security Kimberly Tam.

She was speaking during Riviera Maritime Media’s ‘Where port security meets cyber security’ webinar. This was held at the beginning of Riviera’s Maritime Cyber Security Webinar Week, in association with Maritime Transportation System – Information Sharing and Analysis Center (ISAC), on 4 August.

Dr Tam, who is also academic lead of the university’s Cyber-Ship Lab, said even having back-up systems may not be secure enough.

“Our world is changing. There is more technology and possibilities to create new crimes, which is increasing cyber attack risks,” she said. There have been “leaps in autonomy and information sharing” that is creating vulnerabilities.

“We have seen cyber attacks on infrastructure, energy networks, ports and on port cranes,” Dr Tam continued. “As there is more remote monitoring with sensors, there are new devices that can be hacked.”

Supervisory Control and Data Acquisition (SCADA) networks are particularly vulnerable to hackers due to their weak defence. “SCADA networks get overlooked by IT specialists,” said Dr Tam.

More worrying for port operators is their inability to detect if there has been an intrusion into their IT, SCADA or IoT networks. Dr Tam said would-be hackers could be snooping inside servers undetected. “Hackers would need a lot of reconnaissance of maritime and port servers,” she explained. “We are unable to see who is inside these networks.”

Port operators may not know the intentions of potential hackers or ransomware until it is too late. Hackers could be inside servers to steal information, feed misinformation about manifests, or to input ransomware. “It is not just smash and grab,” said Dr Tam.

With more IoT application in ports, vulnerability of operational technology (OT) to cyber threats is increasing, reducing the air gap between this technology and the connected network. Dr Tam warned these trends lower the security within OT to cyber threats.

If port operators introduce redundancy into IT and OT this could improve security and recovery after an intrusion. “But if this redundancy is too similar, they will have the same vulnerabilities,” said Dr Tam.

University of Plymouth is researching appropriate risk assessment for cyber and cyber-physical systems in maritime and in ports. It is looking at IT and OT systems, with the “aim of giving people information critical for cyber safety and cyber resilience in this sector”, said Dr Tam.

“We are looking at specific case studies for cyber security at ports and we are talking to many in the cruise, container and oil sectors.” The university is considering the plausibility of attacks, calculating realistic risks and the cost of a port cyber attack.

University gains US$3.9M funding for bridge system assessment platform

University of Plymouth’s maritime cyber threat research group’s Cyber-Ship Lab project has made significant progress since it secured £3M (US$3.9M) combined Research England and industry funding in January.

It is creating a unique platform to reproduce any ship’s bridge systems – in service or under development – to assess their cyber risk.

This project has 20 partners on board. More are expected to follow as the research group has gained 150 additional expressions of interest from shipbuilders, maritime IT and operational technology manufacturers, classification societies and insurers.

Named partners include BMT UK, BT Ventures, Eaton, Hensoldt UK (formerly Kelvin Hughes), Altran Group’s Information Risk Management and Lloyd’s Register’s Nettitude.

This project is in the design and build phase. This involves acquiring an extensive and comprehensive collection of in-service or under-development ships’ bridge equipment such as voyage data recorders, radars, automatic identification systems, ECDIS, firewalls, switches, and uninterruptable power supplies.

Various partners have committed to, or are in discussions about, providing their experts’ time or real-world datasets to populate the Cyber-Ship Lab platform.

The group has secured an additional £160,000 (US$207,843) MarRI-UK funding for its Maritime Cyber Risk Assessment framework (MaCRA) work. This has progressed to the market validation stage of the UK Government’s Department for Digital, Culture, Media & Sport’s cyber security academic start up accelerator funding competition, Cyber-ASAP.

Meanwhile, as part of its Cyber-MAR project involvement, the research group is progressing complementary cyber–range work with specialised European container port authorities, enabling them to assess cyber risk and build threat resilience.

Source: rivieramm

In the digital age, information security and data safety issues are critically important. Even large IT companies that are developing complex software and hardware solutions, Internet platforms and IoT (Internet of Things) devices often cannot provide the required level of cybersecurity. Everyone is aware of the latest cases of information leakage and hacking of the protection of such companies as Twitter, Garmin, Intel and other huge industrial players, which were attacked in 2020. And this has an impact on us all, because we or our friends and relatives can be users of any of these products.

Cybersecurity has a huge potential to affect the safety of the crew, vessel, cargo and even ports. Cybersecurity is concerned with the protection of IT systems, onboard hardware and sensors and data leak from unauthorised access, manipulation and disruption. Cybersecurity policies and plans cover different types of risks like information integrity, system and hardware availability on board and in the office of the shipping company. Different incidents can be as the result of:

Problems with data transfer from the shipping company to the vessel and vice versa. For example, incorrect transfer of charts from the shipping company to vessel’s ECDIS can cause delay in voyage or even possibility to reset all charts already installed on ECDIS
Problems with onboard equipment and hardware. Not every member of the crew knows what to do with every operational equipment installed on board in case of disruption or even disaster. That can lead to more heavy consequences with vessel operations
Loss of or manipulation of external sensor data, critical for the operation of a ship. Not to tell about the problems that may occur if vessel systems or shipping company systems will be attacked by hackers.

These are just examples of what can happen with the systems of the ship and the shipping company. With the development of information technologies in maritime logistics, such problems will arise more often if measures are not taken to prevent them in advance.

Cyber Risk Management should:

Define the roles and responsibilities of users, key personnel and management both ashore and aboard
Identify systems, assets, data and capabilities that, if breached, could pose a threat to the operations and safety of the ship
Implement technical and procedural measures to protect against cyber incidents and ensure business continuity
Carry out activities to prepare for and respond to cyber incidents.

The company’s Cyber Risk Management plans and procedures should complement the existing security risk management requirements of the ISM Code and the ISPS Code. Cybersecurity should be seen at all levels of the company, from top management onshore to onboard personnel, as an integral part of the safety culture required for the safe and efficient operation of a ship.

Vessels are increasingly integrated with onshore operations as digital communications are used to conduct business, manage operations, and keep in touch with office managers. In addition, critical vessel systems required for the safety of navigation, power supply and cargo management are increasingly digitized and connected to the Internet to perform a wide range of legitimate functions, such as:

Monitoring of engine operation
Service and management of spare parts
Loading, handling, crane, pump control and laying planning
vessel performance monitoring.

It is important to protect critical systems and data with multiple layers of safeguards that address the role of people, procedures, and technology to:

Increase the likelihood of detecting a cyber incident
Increase the effort and resources required to protect information, data or the availability of IT hardware.

Connected hardware on board should require more than one technical and / or procedural protection. Perimeter defenses such as firewalls are important to prevent unwanted intrusion into systems, but may not be sufficient to combat internal threats.
This defense in depth approach encourages a combination of:

Physical safety of the vessel in accordance with the ship security plan (SSP)
Network protection, including efficient segmentation
Intrusion detection
Periodic scanning and testing of vulnerabilities
Software whitelist
Access and user controls
Appropriate procedures regarding the use of removable media and password policies
Staff awareness of the risks and familiarity with the relevant procedures.

But how important is cybersecurity in the maritime industry?

Marine Digital Fuel Optimization System is a cloud-based system hosted at Amazon facilities in compliance with cybersecurity requirements.

AWS IoT Core provides automated configuration and authentication upon a device’s first connection to AWS IoT Core, as well as end-to-end encryption throughout all points of connection so that data is never exchanged between devices and AWS IoT Core without a proven identity.

AWS IoT Device Defender audits device-related resources (such as X.509 certificates, IoT policies, and Client IDs) against AWS IoT security best practices (e.g., the principle of least privilege or unique identity per device), continuously monitors our device fleets to detect any abnormal device behavior that may be indicative of a compromise by continuously monitoring high-value security metrics from the device and AWS IoT Core (e.g., the number of listening TCP ports on your devices or authorization failure counts).

Case study of data protection and cyber security by Marine Digital

Cyber risk management approach in shipping

The importance of cybersecurity in the maritime industry

Marine Digital FOS box (hardware part, which installed on a vessel) consists of a Data Collection Unit (DCU), a power supply, and a GSM modem, all-in-one robust enclosure, interfacing with the sources of input signals via a read-only NMEA connection, that pulls in data integrated sources, encodes and records it to the integrated storage, and then uploads the collected data to the cloud data lake when a GSM connection is available, autonomously from the shipboard systems. So there is no way to access the equipment on board.

Source: marine-digital

Development of Autonomous ship technologies in Korea compared to Europe.

It is undeniable that Korea is a leading country in the shipbuilding industry. After Hyundai Heavy industries entered shipbuilding in 1968, Korea got ahead of Japan becoming the 1st in the global shipbuilding industry and rose to the number five spot as a maritime powerhouse. Nonetheless, it is said that the technology development of autonomous ships in Korea is about 5 years behind compared to Europe.

Many companies around the world are working on maritime autonomous surface ships, among which Kongsberg and Rolls-Royce seem to be more ahead of others. As Norway’s Kongsberg Maritime acquired Rolls-Royce Commercial Marine in April 2019, they are now fully integrated and the autonomous shipping projects are being conducted under a new organization.

Korean technological innovation toward autonomous ships

Recently, the Korean government announced $132 million will be spent on developing autonomous sailing technology for six years, to achieve the goal of commercializing oceangoing ships that meet level 3 autonomous navigation defined by the International Maritime Organization (IMO).

To realize a fully unmanned autonomous ship soon, the technology development of autonomous vessels such as intelligent navigation system, instrumental automation systems, communication systems, and land operation management system is required and best combined to allow a vessel to operate safely.

Three big shipbuilders in Korea – Hyundai Heavy Industries, Samsung Heavy Industries, and Daewoo Shipbuilding and Marine Engineering – have already set out to the sector, and their current autonomous ship solutions can reach the first stage of ship autonomy.

However, none of them have yet to reach the stage of remotely controlled ships. Most important about autonomous ships is how to combine maritime ship equipment with information, communication technologies with operational technology ensuring cyber-security and establishing massive infrastructure, where autonomous ships will operate with smart docking systems at ports, other maritime facilities.

What technology is needed and how can it be best combined to allow a vessel to operate autonomously?

The technology needed to make a vessel operate autonomously consists of three main parts, ship control systems, digital connectivity from ships to shores, and onshore infrastructures. The first one concerns what the vessels run autonomously. Subsystems such as sensors, positioning systems, other technologies can detect obstacles on a voyage should reliably, securely function. The data gathered from sensors are jointly collected, what is called sensor fusion, and goes back into the vessels’ autonomous navigation system to make decisions based on it. This is a part of the integration occurring of information, communication technologies and operation technology. Many experts worry that autonomous ships can be hijacked as this system is vulnerable to hacking, putting stakeholders at risk.

The autonomous navigation of vessels is similar to self-driving cars in terms of scanning surrounding and detecting obstacles using vehicle sensors like camera, radar, and lidar. However, it is also different than a self-driving vehicle in that every vessel in a certain size is tracked. monitored under the Automatic Identification System (AIS), an automatic system using transceivers on ships, which provides much more information for ship autonomous navigation systems than is available to cars. Vessels sailing on the open oceans also go slower than cars.

For an autonomous ship to auto berth and cross, many sensors on the ships which interact with the main system can allow the ship to dock without crews on board. Even when a ship on this technology, however, is fully operated without crews, it should be connected to a control station, where humans would remotely monitor the ships and their sensors and should be able to take control manually for security as well. Moreover, full autonomy is not the first stage, we would reach middle levels of automation before going fully unmanned.

The Korean government is supporting the project for maritime transport, where a vessel can be controlled remotely when the crew on board first. Even this partial automation can help reduce costs and ease the burden of maritime companies in shortage of laborers. The labor shortage has been a known issue in the shipping industry as it is hard to find qualified employees. So, automation, whether full or partial, can help fill the gap of shortage.

However, this probably means that the technology requires new workers to become more qualified. Although a study of the social impact shows that workers could lose their jobs in several maritime areas due to automation, new jobs can simultaneously be created such as controlling MASS remotely due to there being a control center.

What will be the potential threats for owners and operators of autonomous vessel in the future?

As the benefits of autonomous vessels are multiple and tempting, a variety of organizations, private or public, within the maritime industry have turned toward autonomy to address impediments associated with ship transportation. Progresses in machine learning, ship sensors, and related technologies are not only making the autonomy of ships increasingly feasible but economically attractive. Autonomous vessels are expected to reduce operating expenditures since costs with their crews, all human support facilities, systems, and storage removed.

However, despite these cost-effective advantages, potential threats associated with cyber-attacks must not be neglected. The risks and vulnerabilities linked to autonomous shipping should be anticipated and properly managed with the related technologies advancing. Increased interconnectivity between vessels and onshore infrastructure also increases potential cyber-attacks on ships. Therefore, it is essential to weigh the cyber-risk contours to rank and mitigate any vulnerabilities. As Operational Technology (OT) systems are increasingly automated, the maritime industry has already witnessed cyber-security incidents which led to ships going off their course.

While the existing ships rely on separate systems for managing OT functionality such as bridge, propulsion, and power control, these systems seem to reach the end of life with new technologies adopted. Maritime company owners and operators have been getting OT systems locally and remotely connected via satellite communications and the internet, leading ultimately to a convergence of IT and OT. Sensors on equipment onboard ships transfer data through communication technology (CT). These new integrated technologies are a double-edged sword, which can enable autonomous systems to operate smoothly but put also the growing automation at a greater risk.

To tackle growing concerns about security threats, IMO has a deadline of 1st January 2021 for Maritime Cyber Risk Management to be addressed in ships’ Safety Management Systems. The main focus of the cyber-security program is to put measures in place to protect both OT and IT. It is estimated that cyber-attacks on the maritime industry operation technology (OT) systems have dramatically increased over the last three years.

As these cyber-attacks can have economic impacts and ripple effects on port infrastructures, it might not be easy for vulnerable ports to be fully recovered through insurance policies after OT systems are attacked. The network connecting traffic controls, cranes, vessel berth systems, and cargo handling systems are currently under threat and will be more venerable to cyberattacks especially after fully or partially autonomous vessels emerge in ports. To make matters worse, unlike IT systems, OT systems are more vulnerable to threat as they don’t have a dashboard which allows operators to monitor the condition of all connected systems. The maritime industry progressing towards more digitalization and increasing the reliance on networked and autonomous systems, more numerous vulnerabilities will keep emerging

Unless systems on vessels are properly managed, a large loophole of new cyber-security for hackers to break into can spring up intimidating. With the maritime industry and its digital exposure getting similar to industrial systems and OT, maritime companies must go faster into the direction of protecting their systems and provide a reliable and safe operating environment from a security perspective. Proactive measures must be developed and applied to OT systems since maintaining effective cybersecurity isn’t just an IT issue but is a fundamental operational imperative.

The headquarters of the KR in Busan ©the Korean Register

How Korea respond to maritime security challenges

In preparation for IMO’s Maritime Safety Committee’s resolution “Cyber Risk Management in Safety Management System (MSC.428 (98))” to come into effect, Korean Register of Shipping (KR) has been working together with major shipbuilders to enhance and support the application and verification of ship cybersecurity rules. KR signed a memorandum of understating (MoU) with Hyundai LNG Shipping to conduct joint research on the application, verification, and development of Guideline for Maritime Cyber Security last year. It also signed MoU with Samsung Heavy Industries (SHI) to conduct a joint study on the “Ship Cyber Security Network Construction and Design Safety Evaluation this year.

KR seems to be leading a maritime digital transformation in Korea. It established its own maritime cyber security certification system providing a cyber security certification service for maritime companies. KR has been known for its extensive work on cyber security measures working on big data platforms and e-certificate systems with industry. Moreover, the Korean Register aims to deliver 10 practical digital technologies before the end of 2020.

Source: maritimekr

Cyber security is a major concern for vessels at sea today. The impact of unauthorized, and even authorized, access to ships’ systems can be catastrophic, potentially resulting in reputational, financial and environmental damage, robbery, piracy or simply malicious interference. These are all distinct risks for an unprotected vessel.

Consider potential cyber risks

Not all threats, of course, may be immediately obvious. While an attack on the main propulsion system that causes the vessel to drift without control will be picked up immediately, navigation and positioning systems can be manipulated to show misleading information, inadvertently guiding the ship into trouble.

As the industry slowly approaches truly autonomous shipping, increased reliance on automated systems heightens concerns about security. Vital systems need to be accessible by authorized personnel but protected against any interference. For this reason, type approval processes for systems designed to protect potentially vulnerable components and systems need to consider how the risks of access, both authorized and unauthorized, can be alleviated.

In its type approval process DNV GL identifies four different security level capabilities in line with the IEC 62443 standard. Security Level (SL) 1, the most basic one, provides protection against casual or coincidental violations. Levels 2 to 4 cover increasingly strict protection levels against intentional violation, depending on sophistication of means and the likely level of resources, motivation and skills of potential offenders. Security Level 4 protects against a highly motivated, highly sophisticated attack.

Maritime cyber security specialist Naval Dome has been working with DNV GL, with both organizations sharing knowledge and expertise to improve security requirements for the maritime industry in general and Naval Dome’s own systems in particular. One of the problems identified was that technicians and manufacturers were able to access on-board systems without the knowledge and approval of the crew, which meant they could potentially infect the systems unintentionally.

Therefore a two-step authorization process was needed for which new algorithms had to be developed to prevent remote access without authorization by a vessel’s senior leadership team. To protect the system it is imperative to verify that the person trying to gain access has the necessary authorization and that every action this person takes is recorded in a secure log to mitigate the risk of an internal attack.

Ram Krishnan, CTO at Naval Dome, explains: “In order to protect against marine cyber threats, Naval Dome has developed a solution that is unique among all other cyber threat solutions, because it is designed to protect from the inside-out. We use our software to protect the system itself, thus blocking the two main vectors of attack – external and internal, since the protection is done on the endpoint (PC/HMI).”

One of DNV GL’s original type approval requirements was that once security logs were saved to disk, they could no longer be changed. However, Naval Dome and DNV GL found that this was not necessarily the most secure way of keeping this data safe. Naval Dome therefore devised a new cloud-based solution in which files and logs can be encrypted and saved for 15 years.

Attack on machinery - DNV GL

Machinery connected to communication networks is especially susceptible to cyber threats. Text image 1 – foto-dock.com, DNV GL

The type approval process

The type approval process starts with an assessment of the equipment and its documentation, including installation and operation manuals, applying DNV GL’s stringent and challenging evaluation principles. This often results in revisions before the next phase, product evaluation and test procedure, can begin.

This first phase can be quite a challenge for vendors. Documentation typically requires revision, which can mean it has to go back and forth a number of times until both parties are satisfied with the outcome. This phase also requires vendors to draft test procedure documents which are then sent to the classification society for revision and approval.

Once all of these files have been assessed and revised as necessary, the process moves on to physical testing. If the vendor opts to have systems tested at DNV GL facilities, the vendor will set up the equipment and test protocols before the testing is carried out. In the case of Naval Dome, software was set up on an ECDIS system at the DNV GL facility in Trondheim. However, vendors also have the option to have independent third-party testing performed by DNV GL experts at their own premises.

In order to protect against marine cyber threats, Naval Dome has developed a solution that is unique among all other cyber threat solutions, because it is designed to protect from the inside-out.
Ram Krishnan ,CTO

The tests

DNV GL’s test procedures are based on marinized versions of the international standards ISA/IEC 62443-4-2 and IEC 61162-460 which comprise seven chapters and cover increasingly stringent levels of security requirements. The tests ensure that cyber security equipment is sufficiently robust to prevent penetration attempts while also assessing aspects such as encryption strength. The process covers:

• Human user identification and authentication
• Unique identification and authentication
• Multifactor authentication for all interfaces
• Access privileges
• Software process and device identification and authentication
• User control and functionality
• System integrity
• Data confidentiality
• Restriction to data flows
• Response time to cyber events
• Network/system segmentation
• Monitoring of events
• Resource availability
• The cyber security software must allow the protected application to run without interference

“The tests are important as they can reveal outdated encryption algorithms which the vendor would need to update,” says Dr Mate J Csorba, Global Service Line Leader at DNV GL Digital Solutions.

The tests include remote access, ensuring that ship systems are accessible to vendors’ technicians and authorized on-board staff, but that protocols are in place to prevent malicious access.

“What we are assessing is the security capability of the product. We check the capability and integrity of features such as firewalling and the configuration of the system,” says Csorba.

Depending on the level of security a system is being type-approved for, the number of requirements in each of the seven chapters will differ. The higher the level, the stricter and greater the number of requirements.

The Naval Dome system proved highly effective in DNV GL’s one-week type approval tests. The testing covered the security of the operational system protected by the Naval Dome solution as well as potential interference with vessel systems. “During testing it was not possible to hack, or take control of, vessel systems, and ultimately the ship. The two-step authorization process as well as network and Wi-Fi access security were tested without being able to compromise the protected marine system,” said Ram.

Security concerns

According to DNV GL, few ships are sailing with adequate security systems. “If all ships were sailing with SL1, that would be better than having no security at all, but sadly they are not,” says Csorba.

Without adequate protection, systems on existing vessels are exposed to threats every time data is transferred from shore to ship, or even when crews or technicians do something as straightforward and routine as updating software, including charts and notices to mariners, directly from a CD, a USB drive or technician’s device.

Systems on older ships can be upgraded but will be difficult to bring fully up to date without retrofitting new systems. DNV GL believes that at least SL3 should be specified for newbuilds. According to the definition, SL3 provides “protection against intentional violation using sophisticated means, extended resources, IACS specific skills and moderate motivation”.

To achieve this level of cyber security protection ‒ or the optimum SL4, which offers similar safeguards to those under SL3 with the addition of high offender motivation equipment ‒ vendors need to fully understand the international standards and participate in appropriate workshops with the type approval organization. These help the vendor gain a full understanding of the type approval regulations and requirements, and the approval authority to understand the equipment. Then both parties can jointly determine the security level the vendor or supplier should achieve.

DNV GL and Naval Dome, currently the only specialists capable of offering an SL4 cyber security solution, were able to demonstrate how relatively simple it is to attack live ship systems. The demonstrations have shown that in the absence of adequate cyber protection, the reported ship position can be shifted and the radar display misled. Similarly, the testing experts were able to turn machinery on and off or disable it, and to override fuel control, steering and ballast systems.

These penetration tests allowed Naval Dome to develop a cyber security product that can protect against all kinds of attacks and meet the SL4 standard. The critical factor in certifying cyber security software at this level is to enable shipping and off-shore facilities to implement cyber security quickly and easily without having to re-certify hardware currently in place. Naval Dome’s cyber security software is loaded onto the existing equipment providing cyber security protection immediately.

DNV GL was one of the first classification societies to recognize the growing threat resulting from increased digitalization in shipping and other industries. Its cyber security type approval was introduced in 2017, with the cyber security class notation “Cyber Secure” added the following year.

Attack on the navigation system. A hacker manipulating the navigation system to indicate an incorrect position could cause a severe accident, such as grounding, with potential loss of life and cargo. Text image 2 – Mariusz Bugno – Shutterstock.com, DNV GL

The Cyber Secure notation has three qualifiers: Cyber Secure (Basic), corresponding to SL1 and intended primarily for existing ships; Cyber Secure (Advanced) for newbuilds, which corresponds to SL3 with specific adaptations for maritime systems; and Cyber Secure (+), which covers additional systems not included in the scope of the other two qualifiers but which can be combined with either of them.

Cyber Secure notations by default cover ten systems: propulsion, steering, watertight integrity, fire safety, ballast, thrusters (other than main propulsion), auxiliary systems, communications, navigation and power generation. Other systems can be addressed under the “+” qualifier subject to risk assessments. Under all parts of the notation, a cyber security management system is required for every ship.
Source: DNV GL

A shipborne radar is mainly used to navigate and avoid obstacles in areas such as port accesses and narrow channels and is becoming an important piece of equipment in modern navigation. It can also provide functions over a long distance, such as ship positioning, shoreline monitoring, and rescue location. In particular, it can function in rainy and foggy weather and other harsh environments continuously (Lisowski, 2019). A shipborne radar is also an effective remote sensing tool that can view the scene in real time and provide auxiliary observations in maritime accidents. In the early 1980s, the International Maritime Organization (IMO) stipulated that radars should be one of the necessary equipment for navigation (Capria et al., 2017). With continuous progress and improvement, shipborne radars have been equipped with the function of automatic radar plotting aids, which can provide the course, speed, distance, and orientation of the target ship. They can also perform the functions of automatic identification system (AIS) target fusion, electronic navigational chart (ENC) overlay display, and network monitoring.

An ENC can identify the type and location of a static target. Canada, the USA, Japan, and other countries have promoted the use of ENCs as the core equipment of ship navigation (Zhou et al., 2005). Countries around the world have successively carried out research on ENCs. The International Hydrographic Organization, International Electrotechnical Commission and other relevant international organizations have steadily released S-57 (Transfer Standard for Digital Hydrographic Data), S-63 (Data Protection Scheme), S-100 (Universal Hydrographic Data Model), and S-102 (Bathymetric Surface Product Specification), among others. At present, an increasing number of companies can produce ENC application products that meet international standards, such as Transas of Russia, Sperry of Norway, Offshore of Canada, Atlas and 7cs of Germany, and Raytheon of the USA.

In the 1850s, some scholars proposed the advantages of data fusion of radars and traditional paper charts; they also proposed methods for manual identification (Home, 1952; Dickson, 1952). The information fusion of shipborne radars and ENCs cannot only effectively compensate for the inability of radars to provide water depth data and judge the type of obstacles but also provide the comprehensive situation of the real static and dynamic targets around the ship, greatly improving navigation safety. For example, Kazimierski and Stateczny put forward a method to fuse shipborne radar and AIS information in Electronic Chart Display and Information System (ECDIS) to avoid collision for navigation safety (Kazimierski & Stateczny, 2015). In addition, the target information monitored by shipborne radars can be displayed in the ENCs of ships and shore-based command centers through data fusion technology and wireless networks, which is convenient for a unified action and collaborative operation (Donderi & Mcfadden, 2003). This information fusion also has the following advantages:

(a) The information fusion of oil spills is conducive to the regular monitoring of law enforcement ships or emergency responses. Network monitoring can effectively assist quick disposal.

(b) In maritime search and rescue, radars also show a strong applicability. It can integrate the target information in distress into an ENC so that the command center can quickly dispatch and carry out rescue operations.

(c) As the sea moves, the coastline changes. It is very important for ship collision avoidance ability to monitor the target information of the continental and island coastlines in real time. However, due to altitude limitations, radars are not fully suitable for detecting the entire island. At this time, it is necessary to overlay the object information identified in the ENC to determine an accurate cruise route.

(d) Because the updating of ENC data is periodic, the fusion of radar data can also verify the data effectiveness of the ENC and hydrographic production.

With the release of the S-57 standard by the IMO in 1995, ENCs have played an important role on the ships (Weeks, 1993; Alexandrov, Draganov & Kolev, 1995). Companies in Germany, the UK, Canada, Russia, and China have attempted to achieve the superposition of ENC and radar images (Kolev, Alexandrov & Draganov, 1995; Tang & Shi, 2008; Yang, Dou & Zheng, 2010; Chang, Lai & Hsu, 2014; Wang & Huang, 2014). For example, in 1994, the Offshore ECDIS-3000 system of Canada realized the image superposition of ENCs and radars to ensure navigation safety in foggy weather. Because of many factors, such as low technology level, poor hardware facilities, the huge amount of information in ENCs, and heavy processing of radar signals, the efficiency of data fusion was low (Cai, 2011). Later, Hamilton (Norwegian) and Transas made continuous breakthroughs to improve the efficiency of data fusion (Liang, 2010). At present, some products can realize the rapid information integration of shipborne radars and ENCs, such as the dKart navigator from the USA (Cao, 2017). However, due to commercial competition, their technology has not been open to public. Transas and Sperry have developed a mature radar signal processing card, which can be directly installed in the PCI slot of a general PC. When a radar signal processing card is connected to the radar’s receiving and transmitting antenna unit, it can directly process the radar video signal, record and track the moving target, and control the transceiver (Wang, 2017). In this way, the radar video signal can be transformed into an image and integrated into an ENC through secondary development.

The total shipborne radar data were mapped to the ENC at the outset, so the processing efficiency was very poor. In the 2000s, many scholars found that only the target identified by the shipborne radar should be integrated into the ENC to improve the efficiency of data processing (Lin et al., 2009; Oh, Kim & Jeong, 2015). We divided radar targets into bright and dark in this paper. The strong reflectors of radar electromagnetic waves are called bright targets, such as ships, islands, and shorelines. Bright targets are often used for navigation and collision avoidance. Nishizaki et al. (2014) tried to automatically segment ship targets through image processing and a cluster analysis using shipborne radar images. The automatic detection accuracy in images with ships was high. However, false positive targets were always detected in images without ships. Hu (2018) divided shipborne images into image blocks after preprocessing. He used a deep learning recognizer to classify radar image blocks and realize the classification of ships, shorelines, sea clutters and backgrounds. This method can achieve a better recognition effect. However, ships and land are bright targets, and wrong classification results were obtained when both existed in the image. Wang et al. (2020) applied an automatic multilevel threshold method to extract bright targets in a shipborne radar image with fine preprocessing. The experimental results of this method are excellent. However, the efficiency is relatively low because of the time required to compute a multilevel threshold. The reflectors that can weaken the radar electromagnetic echo are called dark targets, such as oil spills. Dark targets are mostly used for monitoring marine disasters. Zhu et al. (2015) proposed a manual single-threshold method to extract oil spills after adjusting the whole gray level of the shipborne radar image. This method can adjust the gray distribution of the sea clutter area, and it has a certain reference significance for dark target extraction. Based on their study, Xu et al. (2019) proposed the application of a contrast enhancement algorithm and local adaptive threshold to extract oil spills, making dark targets easier to identify.

In this paper, we propose a dual-threshold method to segment bright targets and the local adaptive threshold to extract dark targets in shipborne radar images. Then, the targets are mapped into the ENC, providing technical support for hydrographic data inspection and disaster monitoring.

Materials & Preprocessing Methods

Materials

32 shipborne radar images were acquired during the cruises of the teaching-training ship Yukun (Fig. 1) of Dalian Maritime University. Among them, 21 images contained ships, 7 images contained shorelines, and 6 images contained oil spills. The three images shown as Fig. 2 are used to introduce our method. The radar parameters are listed in Table 1. The data acquisition period is 2 s. The monitoring radius in the images is 0.75 nautical miles (NM). The image size was 1,024 × 1,024 pixels and the gray level was 256. The chart of the Port of Dalian is used here, as shown in Fig. 3. The image processing and analysis platform is MATLAB 2014a, and the system development platforms are Visual Studio 2010 and ArcGIS 10.2.

Data preprocessing

Many oil films exist around the ship, as shown in Fig. 2A. Ships and coastlines information are presented in Figs. 2B and 2C. The images needed to be denoised first, especially the co-frequency interference noises, which seriously affect the uniform gray distribution. The data preprocessing scheme is shown in Fig. 4. First, the image was converted to a Cartesian coordinate system. After that, the co-frequency interference noises were segmented by the Laplace operator and Otsu method. They were then suppressed via mean filtering.

Figure 1: Hardware architecture.

(A) Installation position of radar antenna. (B) Radar Image Acquisition System. (C) ENC.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-1

The original shipborne radar image was collected in the Cartesian coordinate system. In Formula Eq. (1), the image is transformed into a polar coordinate system for understandable, as shown in Fig. 5.(1) $\{\begin{matrix} ρ = \sqrt{x^{2} + y^{2}} \\ θ = arctan (\frac{y}{x}) \end{matrix},$

where x and y are the abscissa and ordinate of the Cartesian coordinate system, respectively, and ρ, θ are the distance and orientation of Polar coordinate system, respectively.

Figure 2: The original radar images.

(A–C) are acquired at 23:20:27 21th July, 2010, 17:27:44 10th August, 2015, and 08:59:20 12th August 2015, respectively.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-2

Many meaningful image features are present in the Cartesian coordinate system. For example, the co-frequency interferences appear as radial noises in the vertical direction (Fig. 6), and they are processed without considering the angle factor. Therefore, we used Formula Eq. (2) to transform the experimental data from the polar coordinate system to the Cartesian coordinate system.(2) $\{\begin{matrix} x = ρ cos θ \\ y = ρ sin θ \end{matrix},$

The Laplace operator is as follows:(3) $\nabla^{2} f = 4 f (x, y) - [f (x + 1, j) + f (x - 1, j) + f (x, y + 1) + f (x, y - 1)],$

where x and y are the abscissa and ordinate of the image, respectively. The mean filtering is calculated (n = 3 in our experiment) as(4) $f (n, y) = \frac{\sum_{i = 0}^{n} X (n - 1, y) + X (n + 1, y)}{2 n},$

Figure 2C is used to introduce the preprocessing flow, as shown in Fig. 7. First, the original radar image was transformed from the polar coordinate system to the Cartesian coordinate system (Fig. 7A). Then, the Laplace operator was used to enhance the co-frequency interference and weaken the other information (Fig. 7B). Next, the Otsu threshold was used to segment the co-frequency interference (Fig. 7C). Finally, a mean filter was used to suppress the co-frequency interferences, as shown in Fig. 7D.

Table 1:

Parameters of shipborne radar.

Parameter	Value
Product type	Sperry Marine B.V.
Band	X-band
Detection distance	0.5∕0.75∕1.5.3.6.12 NM
Image size	1,024 ×1,024
Antenna type	Waveguide split antenna
Polarization mode	Horizontal
Horizontal detection angle	360°
Rotation speed	28–45 revolutions/min
Length of antenna	8 ft
Pulse repetition frequency	3000 Hz/1800 Hz/785 Hz
Pulse width	50 ns/250 ns/750 ns

DOI: 10.7717/peerjcs.290/table-1

Figure 3: Chart of Dalian Port.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-3

Figure 4: Flow chart of data preprocessing.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-4

Figure 5: Transformation of coordinate system.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-5

Figure 6: Co-frequency interferences in two coordinate systems.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-6

Figure 7: The preprocessing method.

(A) Coordinate system transformation. (B) Laplace operator convolution. (C) Otsu automatic threshold segmentation. (D) Mean filter.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-7

Results

Data analysis process

The data analysis process is shown in Fig. 8. First, it is necessary to determine whether to extract the light or dark target firstly. For a bright target detection, the dual-threshold method is used for segmentation after setting the monitoring area. Otherwise, the local threshold method is used to segment the dark target after setting the effective wave area. Then, the identified targets are transformed to the polar coordinate system. After that, the contour of the target is extracted, and the boundary points of the contour are projected onto the geographic coordinate system for data fusion.

Bright target segmentation

In the original shipborne radar image, the brightness of the wave echo area at a distance is high. Therefore, the monitoring area should be set before identifying the ship or coastline. We removed the image information within 0.3 NM (mainly the highlighted wave echo pixels) and retained the image information of the bright target monitoring area, as shown in Fig. 9.

The following dual-threshold method is selected to extract the target, as shown in Fig. 10.(5) $D (x) = \{\begin{matrix} d (x) > T (g r a y) \\ A (r e g i o n) > T (A r e a) \end{matrix},$

Figure 9: Monitoring area is set between 0.3–0.75 NM.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-9

Figure 10: The dual-threshold segmentation.

(A) Gray threshold segmentation of T (*gray*) = 120. (B) Pixel number threshold segmentation of *T(Area)* = 20. (C) Composite image.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-10

where D (x) is the continuous pixel of the target, d (x) is the gray value of the image pixels, T (gray) is the gray threshold, A(region) is the number of pixels of the continuously highlighted image regions, and T(Area) is the number threshold of A(x). For a radar image of 1,024 pixels ×1,024 pixels with a monitoring range of 0.75 NM, the actual area of 1 pixel is 7.34 m². It is suggested to set the continuous highlight area of less than 20 pixels as speckles and eliminate them here. If the radar range increases and the image size does not change, then T(Area) should be smaller.

Dark target segmentation

Because of the different reflection characteristics of the monitoring targets, the identification of oil spills is more complicated than that of ships and coastlines. The oil film is dark in the sea clutter area, so the effective monitoring of the wave should be determined first.

In the image after preprocessing, the information is deleted at a long distance (0.5 NM away in our experiment). Then, the image gray distribution matrix (Fig. 11B) is generated by convolution of a 20 ×80 window matrix. Next, the threshold (“61.02” in Fig. 11C) is selected in the color scale image of Fig. 11B to determine the effective wave range by visual interpretation and manual selection , as shown in Fig. 11D.

The method of final segmentation is also more complicated than that of bright targets. Oil films can suppress the echo of sea waves and display them as relatively dark areas on waves in the images (Tong et al., 2019; Fingas & Brown, 2018). Therefore, local thresholds and active counter models are often used for segmentation (Liu et al., 2019). Figure 11D is segmented as Fig. 12 by the local threshold method (Xu et al., 2019) as:(6) $T = m \times [1 + k (\frac{v}{R^{2}} - 1)],$

where m is the local mean, k is a user-defined parameter that takes negative values, v is the local standard deviation, and R is the dynamic range of the standard deviation. In our experiments, k = 0.25 and R = 128.

Data fusion

The image acquisition method of shipborne radars is head-up, and the ENC in our experiment is north up. To match the radar target with the ENC, the identification result needs to be rotated. An image matrix coordinate system was thus established to reduce the difficulty of image rotation, as shown in Fig. 13.

Figure 11: The effective wave range is set near the own ship.

(A) Preprocessed image. (B) Gray distribution matrix. (C) Manual threshold selection in color scale image of (B). (D) Determination of effective wave range.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-11

Figure 12: The local threshold segmentation.

(A) Rough result of oil spill. (B) The speckles are removed.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-12

Figure 13: Image matrix coordinate system of Fig. 10B.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-13

The image rotation formula is:(7) $\{\begin{matrix} x^{'} = x c o s (a) - y s i n (a) \\ y^{'} = x s i n (a) + y c o s (a) \end{matrix}$

where x′ and y′ are the abscissa and ordinate after rotation, respectively; x and y are the abscissa and ordinate before rotation, respectively; and a is the angle of rotation.

The identification target is rotated with the angle according to the heading of the ship, and the counter is extracted as shown in Fig. 14.

Figure 14: The counter extraction of coastline target.

The image is rotated with heading direction (−19.5°).

Download full-size image

DOI: 10.7717/peerjcs.290/fig-14

The longitude and latitude coordinates of the ship can be obtained through GPS. In ArcGIS, the plane coordinate (X_os, Y_os) of the ship in meters can be obtained by projecting it from the WGS_1984 geographic coordinate system to the Beijing_1954 projection coordinate system. Based on the image matrix coordinates (X_Pic, Y_Pic), radar detection distance D, image radius R, and plane coordinates of the ship (X_os, Y_os), the plane coordinates (X_BJ, Y_BJ) of the target contour boundary point in the geodetic coordinate system of Beijing_1954 can be obtained as(8) $\{\begin{matrix} X_{B J} = X_{o s} + (X_{P i c} ∕ R) D \\ Y_{B J} = Y_{o s} + (Y_{P i c} ∕ R) D \end{matrix}$

Then through ArcGIS, target contour boundary points were converted back orderly to the geographic coordinates of the WGS_1984 coordinate system, as shown in Table 2.

After the geographical coordinates of the target boundary points were obtained, the target polygons were generated into an ENC based on ArcGIS Objects, as shown in Fig. 15. The detailed steps include the following: (a) According to the interfaces of ISpatialReferenceFactory and ISpatialReference, the properties of spatial reference (SpatialReference) and spatial projection (Project) of the target contour boundary point (IPoint) are set. (b) Using the AddPoint method, the boundary points are added to the point set (IPointCollection) to generate the target polygon. (c) The geometry attribute of the target polygon is assigned to the interface of the IElement. (d) Through the interface of IFillShapeElement, method of AddElement, class of GraphicsContainer and class of ActiveView, the target polygon is was added to the development control (axMapControl) of the ENC.

The coastline targets were fused into the ENC, as shown in Fig. 16A, which includes local information before and after fusion. The oil spill targets and ship targets were fused into the ENC, as shown Figs. 16B and 16C, respectively.

Discussion

Role of data fusion

The target data fusion between radars and the ENCs cannot only verify the hydrographic data but also assist the emergency command of marine disasters and accidents. Bright targets can be used to test the accuracy and timeliness of hydrographic data. For example, Fig. 16A can determine whether the island coastline data of the ENC are correct or whether the location of the lighthouse is correct. The coastline changes due to the influence of tides and storm surges. The island profile of low tide is shown in the ENC (green region in Fig. 16A). The island belongs to a strong reflector, so the radar can only detect its illuminated surface but not the blocked area. The radar detected that the coastline of the irradiating surface was beyond the island profile of the ENC. Thus, the ENC data must be updated in time.

Table 2:

Geographical coordinates samples of coastline target boundary point.

The GPS position of own ship is 122.0358°, 39.0319°.

Point ID	Longitude (°)	Latitude(°)
1	122.02595	39.03030
2	122.02598	39.03030
3	122.026015	39.030328
4	122.026015	39.030352
5	122.026014	39.030376
6	122.026046	39.030377

DOI: 10.7717/peerjcs.290/table-2

Figure 15: Object model diagram of space polygon generation in ArcGIS Objects.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-15

Figure 16: Information fusion.

(A) Coastlines. (B) Oil spills. (C) Ships.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-16

Dark targets can be used for marine disaster monitoring, such as the monitoring of oil spills, as shown in Fig. 16B. Remote sensors have become the main means of marine disaster monitoring technology. With respect to radar applications, airborne and spaceborne radars are maturely used for offshore disaster monitoring (Sankaran, 2019; Hammoud et al., 2019; Gallego et al., 2018). However, spaceborne remote sensing cannot conduct monitoring in real time. Airborne sensors cannot be used in severe weather conditions. The technology of shipborne radar marine disaster monitoring can overcome adverse weather conditions and remotely monitor real-time situations with high resolution. The data fusion of dark targets can provide technical support for the early warning and emergency disposal of marine disasters and accidents.

The ship target fusion of shipborne radars and ENC can provide a more effective guarantee for navigation safety. Our experimental data range is 0.75NM, as shown in Fig. 16C, which only provides a ship target fusion method for radars and the ENC. Real applications need to improve the detection range of radar images.

Different segmentation methods for bright and dark targets

Ship and coastline targets have strong reflectivity, as highlighted in the images. Fixed thresholds can be used to segment them. However, oil film extraction is more difficult. A single threshold cannot identify discrete oil films in different positions. The local adaptive threshold is a better solution. In addition, because the database of the oil spill analysis is composed of waves, the monitoring range is close to the own ship, as shown in Fig. 11D. But, ships and coastlines are monitored for the inspection of hydrographic data. The monitoring area is located far away from the ship (Fig. 9).

Comparison with other bright target segmentation methods for shipborne radar images

The method of Wang et al. (2020) (hereafter referred as Method 2) and our method for extracting bright targets are compared here. Figure 2B and 2C are segmented using Method 2, as shown in Fig. 17.

Figure 17: Rough bright targets extraction of Figs. 2B and 2C by Method 2.

(A) and (B) are 0.3–0.75 nm and 0.5–0.75 nm monitoring area of Fig. 2B , respectively. (C) and (D) are 0.3–0.75 nm and 0.5–0.75 nm monitoring area of Fig. 2C, respectively.

Download full-size image

DOI: 10.7717/peerjcs.290/fig-17

Compared with Fig. 10A, only the preliminary extraction effect presented in Fig. 17B is similar. Too many false positive targets are present in Figs. 17C and 17D, implying that Method 2 is not suitable for large bright-target detection, such as islands and coastlines. Many false positive targets also exist, as shown Fig. 17A. For the complexity of the original image of the shipborne radar, Method 2 needs to set a smaller monitoring area to obtain good segmentation results for ships, as shown in Fig. 17B. Our method can identify ships and islands correctly, and set a large monitoring area, which has strong applicability in bright-target identification.

Comparison with other dark target segmentation methods for shipborne radar images

The methods of Zhu et al. (2015) (hereafter referred as Method 3), Xu et al. (2018) (hereafter referred as Method 4) and our method for extracting oil spills are compared with Fig. 2A in Fig. 18.

Figure 18: Comparison with other oil film segmentation methods of Fig. 2A.

(A) our method. (B) Method 3 (T_gray = 100). (C) Method 4 (T_gray = 100, T_area = 30).

Download full-size image

DOI: 10.7717/peerjcs.290/fig-18

In Method 3, a global threshold method is proposed for oil film identification after gray adjustment, as shown in Fig. 18B. From the perspective of an expert interpretation, the recognition effect presented in Fig. 18B is not as good as that of our method. As the global threshold increases, Method 3 recognizes the false oil films in the area without wave information. In Method 4, a dual-threshold method is proposed to detect oil spills after choosing the wave range, as shown in Fig. 18C. Although a small wave range is selected, as shown in Fig. 18C, the segmentation effect is not good. As the wave range increases, the segmentation result becomes worse. Therefore, the major drawback of Method 4 is that it cannot select a further effective wave monitoring range. Therefore, the local adaptive threshold method is more suitable for the segmentation of large-range oil films and yields better results.

Guarantee of navigation safety using the data fusion of an ENC and shipborne radar

The production of an ENC is time-consuming and has a slow update speed. Before sailing, the captain and pilot will create a general route using the information of the ENC. However, in the actual driving process, owing to the weak current situation of the ENC, the route will be made local adjustments in accurate information acquired by the radar.

The ENC can provide information on sunken ships, reefs, and shoals, among others. Shipborne radars can provide collision avoidance information for all other ships around the ship. After the data fusion of the ENC and shipborne radars, the dynamic information of a wider-range radar monitoring and the static information provided by the ENC can form a more efficient digital platform, which can improve the monitoring ability of pilot, and provide better guarantee for navigation safety.

Lack of AIS data

At present, ENCs and shipborne radars have been fused with AIS system data. In radars and ENCs, the location and attribute information of a ship with AIS can be obtained (Jung, Kim & Song, 2007). However, the installation of AIS equipment is not required for small fishing boats or maintenance boats. Radars have an obvious marking effect on strong reflectors, which can highlight their reflecting surfaces on the screen. Therefore, the role of radar navigation and collision avoidance is essential. In addition, AIS targets use specific symbols in radars and ENCs, and there is no real shape. If the radar installation position of the ship is much higher than that of the target, then the target contour can be identified completely and integrated into the ENC. This characteristic makes up for the deficiency that AIS data cannot obtain the contour of the targets.

Conclusions

Using the original shipborne radar images, this paper proposes methods for target extraction and fusion with ENCs for hydrographic data inspection and disaster monitoring. The experimental results show that proposed dual-threshold method can extract bright targets quickly and accurately. The local threshold method is more suitable for the complex image segmentation of dark targets. Finally, a data fusion method is proposed for shipborne radar images and ENC, which can be effectively used in hydrographic data inspection and marine disaster data monitoring.

It is noteworthy that the radar images used in this study have a relatively short range, which can meet the requirements of short-range hydrographic data inspection and auxiliary oil spill clean-up. In future work, we will increase the collection of radar images in a wider-range and improve the applicability of our method.

Source: peerj

Newer Posts

Older Posts

Onboard experiments

Journey time

Covid-19 disruptions on manned research vessels

The future of autonomous shipping

Finance Crunch

Big Ships, Small Ships

But how important is cybersecurity in the maritime industry?

Case study of data protection and cyber security by Marine Digital

Cyber risk management approach in shipping

Development of Autonomous ship technologies in Korea compared to Europe.

Korean technological innovation toward autonomous ships

What technology is needed and how can it be best combined to allow a vessel to operate autonomously?

What will be the potential threats for owners and operators of autonomous vessel in the future?

How Korea respond to maritime security challenges

Materials & Preprocessing Methods

Materials

Data preprocessing

Figure 1: Hardware architecture.

Figure 2: The original radar images.

Figure 3: Chart of Dalian Port.

Figure 4: Flow chart of data preprocessing.

Figure 5: Transformation of coordinate system.

Figure 6: Co-frequency interferences in two coordinate systems.

Figure 7: The preprocessing method.

Results

Data analysis process

Figure 8: Data analysis process.

Bright target segmentation

Figure 9: Monitoring area is set between 0.3–0.75 NM.

Figure 10: The dual-threshold segmentation.

Dark target segmentation

Data fusion

Figure 11: The effective wave range is set near the own ship.

Figure 12: The local threshold segmentation.

Figure 13: Image matrix coordinate system of Fig. 10B.

Figure 14: The counter extraction of coastline target.

Discussion

Role of data fusion

Figure 15: Object model diagram of space polygon generation in ArcGIS Objects.

Figure 16: Information fusion.

Different segmentation methods for bright and dark targets

Comparison with other bright target segmentation methods for shipborne radar images

Figure 17: Rough bright targets extraction of Figs. 2B and 2C by Method 2.

Comparison with other dark target segmentation methods for shipborne radar images

Figure 18: Comparison with other oil film segmentation methods of Fig. 2A.

Guarantee of navigation safety using the data fusion of an ENC and shipborne radar

Lack of AIS data

Conclusions

News

Quick Menu

Company DETAILS

ISO 9001:2015 CERTIFIED