Pilots fail in two ways, and the second is considerably more expensive than the first. The visible failure is a pilot that does not work: the crew abandon it, the data is thin, and somebody has to explain a wasted quarter. That outcome is cheap and informative. The dangerous failure is a pilot that succeeds and then does not scale — and analysis of enterprise proof-of-concept work names it precisely as the most dangerous scenario, because when a system performs well in a controlled trial, executives assume the hard work is done. The demonstration worked, the numbers looked good, and nobody noticed that the pilot ran on data manually prepared by the project team, without the integrations the real environment requires. Translate that to a fleet and the picture is familiar: the best-run vessel, the strongest crew, an equipment register somebody tidied specially, the vendor's consultant answering within minutes, and everyone aboard aware they are being watched. That pilot will succeed. It will also tell you almost nothing about the other eleven ships. The research on why pilots stall is consistent about the root cause too: unclear success metrics, and poorly defined problems with no measurable criteria. Start a free trial of Marine Inspection and write the criteria before the pilot rather than after it.
Visible failure
The pilot does not work
Crews stop using it, entries dry up, the data is thin. Uncomfortable to report and genuinely useful — you have learned something real about the product, the configuration or your own readiness, at the cost of one vessel and a few weeks.
Cheap. Informative. A legitimate result.
Dangerous failure
The pilot succeeds and does not scale
It ran on prepared data, on the best vessel, with the strongest crew, with the vendor on call, with everybody watching, and without the integrations or the ordinary conditions the rest of the fleet operates under. The board approves a fleet rollout on the strength of it, and the second vessel behaves nothing like the first.
Expensive. Misleading. Approved on evidence that did not transfer.
The design instruction that follows is counter-intuitive and worth stating plainly: make the pilot as unfavourable as it can be while remaining a fair test. Every advantage you grant it is a question you have chosen not to answer, and you will pay for that question later at fleet scale rather than now at single-vessel scale.
Write the Charter Before Anything Is Installed
Guidance on escaping stalled proof-of-concept cycles converges on one mechanism: a pre-implementation document that forces objective criteria into the open before work begins. Book a Marine Inspection demo and produce this page before agreeing any dates.
01
The business problem, in one sentence
Not "improve maintenance visibility". Something falsifiable — records are made hours after the work and cannot be trusted, or we cannot produce a component history at survey, or we discover certificate expiries too late. Analysis of stalled pilots identifies poorly defined problems with no measurable criteria as the primary cause of failure.
02
The executive sponsor and the project lead, named
Two different people, both named, with the second having stated time for it. A pilot with no sponsor has nobody to unblock it and nobody to end it, which is how a trial becomes permanent without ever being approved.
03
Baseline metrics, measured before you start
The single most commonly skipped element and the one that makes proof possible. If you do not know how long a job closure currently takes, how many findings a vessel currently raises, or how long a component history currently takes to retrieve, you cannot demonstrate any improvement afterwards — only assert one.
04
Success criteria, with numbers attached
Written before the pilot and specific enough that two people would agree afterwards whether they were met. Unclear success metrics are repeatedly identified as a reason projects stall between pilot and production, and criteria agreed after the fact are opinions rather than tests.
05
The evaluation date, fixed
Not a duration that might extend. A date on which a decision is made. Open-ended exploration without a fixed evaluation date is described as the structural mechanism that produces stalled pilots — initiatives neither cancelled nor scaled, consuming resources and credibility while delivering neither transformation nor clarity.
Design It to Be Hard
Every favourable condition you give a pilot is a question you have deferred to fleet rollout, where it costs eleven times more to answer. Sign up for Marine Inspection and choose the awkward vessel deliberately.
The vessel
Not the flagship. A ship with mixed record quality, awkward equipment, a mixed-nationality crew and a demanding trading pattern. If it works there, the fleet argument is already made. If it works on your best ship, you have learned that your best ship is your best ship.
The data
Load it as it actually is rather than as somebody cleaned it for the occasion. A pilot running on curated data prepared by the project team is the textbook example of a trial that succeeds and does not transfer, because the real environment has never had that treatment.
The support
Use the support the fleet will actually get, at the response times the contract actually offers. A consultant answering in four minutes during a pilot and in two days afterwards has produced an adoption result you cannot reproduce.
The observation effect
Crews behave differently when they know they are the pilot, and you cannot eliminate that entirely. What you can do is run long enough for novelty to wear off, and weight the later weeks more heavily than the first — because week one measures enthusiasm and week five measures habit.
Measure the before, or you will only ever be able to assert the after
Time a job closure on paper today. Count findings raised on that vessel last quarter. Ask a second engineer to retrieve the last overhaul on a specific pump and time that too. Count how many completions in the current system carry a measured value rather than a tick. Four measurements, one afternoon, and without them the pilot produces impressions rather than evidence — which is precisely the condition that leaves initiatives stalled between trial and production.
How Long, and Why the Clock Is Different at Sea
General guidance points at four to six weeks. Maritime has a constraint that overrides it, and ignoring that constraint is how pilots die without a decision being made. Schedule a walkthrough and set the date against the vessel's rotation rather than against a calendar quarter.
The general recommendation
A bounded pilot with a fixed timeline in the region of four to six weeks, with one approach suggesting you set the worst-case expectation at eight weeks while internally targeting four — creating buffer while typically delivering ahead of schedule. The compression is deliberate: it creates urgency, prevents scope creep and forces decisions.
The maritime override
The pilot must conclude before the crew rotates, or the result is contaminated by a handover you did not design for. An open-ended pilot at sea does not drift indefinitely the way a shore-based one does — it dies quietly at the first crew change, when the people who learned it leave and nobody has decided anything.
So the working rule
Start mid-rotation with a settled crew who have at least six weeks left aboard, and set the evaluation date inside that window. That gives enough time for novelty to fade and habit to form, and it means the decision is taken by people who used the system rather than by people who inherited it.
And the handover is a second test
If the pilot vessel changes crew shortly after the evaluation date, watch what happens. A practice that survives one rotation with no shore intervention is genuinely embedded; one that collapses tells you the fleet rollout needs handover material rather than more training. That is a free and extremely informative extra test.
What to Measure
Nine measures, each with a baseline taken beforehand, each answerable from the system or from a stopwatch rather than from an opinion. Start a free trial and capture the left column before installing anything.
Table 1: Pilot Measures, With Baselines
| Measure | Baseline to take first | What the pilot should show | Why it matters |
|---|---|---|---|
| Job closure time | Time a real closure in the current system, at the machine | Faster, measured the same way by the same person | If it is slower, nothing else on this list will hold |
| Distinct users | How many people currently make maintenance entries | More, and spread across the department rather than one officer | Capture at the work is the whole mechanism |
| Entry timing | When entries are currently made relative to the work | Timestamps matching watch and round patterns | Everything logged at one hour means records are still reconstructed |
| Measured values present | What share of current completions carry a number | A materially higher share on jobs that call for one | The difference between a record and a tick |
| Findings raised | Count on that vessel over the last quarter | Higher, and sustained rather than a first-week spike | Rising reporting is a healthy signal, not a worsening one |
| Findings closed with evidence | What share of current findings have documented closure | Most, with evidence attached rather than a status change | This is what an auditor or a buyer's superintendent reads |
| History retrieval time | Time a second engineer finding the last overhaul on a named unit | Seconds, at the machine, without asking anybody | The question engineers actually ask, and a strong adoption predictor |
| Offline share | Not applicable — this is a new capability | A meaningful proportion of entries created with no connectivity | If everything is created online, people are still waiting to be somewhere convenient |
| Crew verdict | Ask a second engineer and a rating what they think of the current process | Faster than what it replaced, said by the same people | The single most predictive measure, and it is not in any dashboard |
Week 1 vs Week 5
Weight the later weeks more heavily than the early ones. Week one measures enthusiasm and week five measures habit, and only the second one tells you anything about the eleven vessels that will not have anybody watching them.
Three Outcomes, and All of Them Are Decisions
Naming the possible endings in advance is what stops a pilot becoming permanent by default. Book a walkthrough and write these into the charter alongside the criteria.
Scale
The criteria were met, on a vessel chosen to be difficult, with data as it actually is and support as the fleet will actually receive it. Proceed to fleet rollout with the pilot's configuration corrections already applied, and with the observed results in the paper rather than projections.
Requires: criteria met, and the conditions honest
Stop
The criteria were not met and the reason is the product or the fit rather than something you can change. Ending cleanly is a good outcome and should be named as one in advance, so that stopping is a decision somebody makes rather than an admission somebody avoids.
Requires: naming this as legitimate before starting
Extend, once, with a reason
The criteria were not met and the cause is identified and fixable — a configuration error, a checklist that was too long, an access model scoped too narrowly. One extension, a new fixed date, and the specific change written down. A second extension without a named cause is how pilots become permanent.
Requires: a named cause and a new fixed date
The reason to write all three into the charter is that the fourth outcome — continuing indefinitely — is the one that happens when none of the others is defined. That state has a name in the wider literature: initiatives neither cancelled nor scaled, consuming resources and credibility while delivering neither transformation nor clarity. In shipping it usually resolves itself at a crew change, which is the worst possible way for a decision to get made.
How Often Pilots Actually Convert
The published figures vary enormously, and the variance itself is the most useful thing about them. Start a free trial and define what production means for you before quoting any of these.
What the numbers say
Reported conversion rates from proof of concept to production range widely across 2026 analysis — one study of enterprise pilots reports four reaching production for every thirty-three launched, a twelve percent rate; another puts successful scaling below twenty-seven percent; a market intelligence figure has the average enterprise abandoning forty-six percent of proofs of concept before production; and vendor-authored analyses run considerably higher still.
Why they disagree
Because production is not defined consistently. Some counts require full-scale deployment delivering measurable return; others count any move beyond the trial. That is the same ambiguity that kills individual pilots — success measured against criteria nobody wrote down in advance — reproduced at the level of the research itself.
What to take from it
Not a percentage to quote in a board paper, since the range makes any single figure indefensible. The usable finding is directional and consistent: most pilots do not scale, and the causes cited are the same across every source — weak business objectives, poor data readiness, absent governance, organisational resistance and unclear success metrics. Four of those five are within your control before the pilot starts.
Common Pilot Design Errors
Each of these produces a result that looks like an answer and is not. Schedule a demo and check the plan against them before committing to a date.
Table 2: Errors That Produce Misleading Results
| Error | What it produces | When you find out | The fix |
|---|---|---|---|
| No baseline | Improvement that can be asserted and not shown | At the approval meeting, when somebody asks compared to what | Four measurements, one afternoon, before installing anything |
| No fixed end date | A trial that continues until a crew change ends it | Never, formally — which is the point | A date in the charter on which a decision is made |
| The easy vessel | A success that does not transfer to the rest of the fleet | On vessel two, after the fleet decision | Choose mixed record quality and a demanding trading pattern |
| Prepared data | A configuration validated against records nobody else has | During the fleet migration, expensively | Load the data as it is; the mess is the test |
| Elevated support | An adoption rate you cannot reproduce at fleet scale | Month two of the rollout | Use contracted response times during the pilot |
| Criteria written afterwards | A debate about whether it worked rather than a test | Immediately, and it never fully resolves | Numbers in the charter, agreed by both sides in advance |
| Only the first week counted | Novelty mistaken for adoption | When the curve declines in month two | Weight later weeks more heavily and run past the novelty |
| Officers only | A trial of shore-side reporting rather than of capture at the work | When entry volumes stay low despite positive feedback | Give access to everyone who does the work, from day one |
| No stop criterion | A pilot nobody can end without it looking like failure | Months later, still running, unmeasured | Name stopping as a legitimate outcome in the charter |
PILOT DESIGN REALITY
The conversion statistics cited here come from analysis of enterprise technology pilots generally, principally artificial intelligence and data projects, rather than from maritime software specifically. Reported rates vary from around twelve percent to below twenty-seven percent depending on how production is defined, with some vendor-authored analyses claiming far higher failure rates; the range is wide enough that no single figure should be quoted as authoritative, and the consistent finding is directional rather than numerical. The four to six week guidance is likewise drawn from general practice and is overridden in shipping by crew rotation, which is why this page recommends setting the evaluation date inside a single rotation rather than to a fixed number of weeks. This page is published by a software vendor, and the advice to run the pilot on a difficult vessel with uncleaned data and contracted support levels makes our own product harder to sell, which is why it is worth following. A pilot that ends in a decision to stop is a successful pilot, and any vendor unwilling to write that into a charter is telling you something.
Frequently Asked Questions
What is the biggest risk in a single-vessel pilot?
A pilot that succeeds and does not scale, which analysis of enterprise proof-of-concept work names as the most dangerous scenario. When a system performs well in a controlled trial, executives assume the hard work is done — but the trial ran on data manually prepared by the project team, without the integrations and conditions the real environment imposes. In a fleet that translates to the best vessel, the strongest crew, a specially tidied equipment register, a consultant answering within minutes, and everybody aware they are being observed. That pilot will succeed and will tell you very little about the other vessels, and the fleet decision gets taken on evidence that did not transfer.
How do you design a pilot that proves something?
Make it as unfavourable as it can be while remaining a fair test. Choose a vessel with mixed record quality, awkward equipment and a demanding trading pattern rather than the flagship. Load the data as it actually is rather than as somebody cleaned it for the occasion. Use the support response times the fleet will actually receive rather than an elevated level for the trial. And run long enough that novelty fades, weighting the later weeks more heavily than the first — because week one measures enthusiasm and week five measures habit. Every favourable condition you grant is a question deferred to fleet rollout, where answering it costs considerably more.
What has to be written down before starting?
A charter, produced before implementation, containing at minimum: the business problem in one falsifiable sentence; a named executive sponsor and a named project lead with stated time; baseline metrics measured beforehand; success criteria with numbers attached, specific enough that two people would agree afterwards whether they were met; and a fixed evaluation date. The last is structural rather than administrative — open-ended exploration without a fixed evaluation date is described as the mechanism that produces stalled pilots, initiatives neither cancelled nor scaled while consuming resources and credibility.
How long should a maritime pilot run?
General guidance points at a bounded four to six weeks, with one approach setting worst-case expectations at eight weeks while internally targeting four, on the basis that compression creates urgency and prevents scope creep. Shipping adds a constraint that overrides the calendar: the pilot should conclude before the crew rotates. Start mid-rotation with a settled crew who have at least six weeks left aboard and set the evaluation date inside that window, so the decision is made by people who used the system rather than by people who inherited it. An open-ended pilot at sea does not drift indefinitely — it dies at the first crew change without anybody deciding anything.
What should we measure?
Nine things, each with a baseline captured beforehand: job closure time, timed the same way by the same person before and after; how many distinct people make entries; when entries are made relative to the work; what share of completions carry a measured value rather than a tick; findings raised, compared with the previous quarter; findings closed with evidence attached; how long a second engineer takes to retrieve the last overhaul on a named unit while standing next to it; the proportion of entries created offline; and what the crew say, asked of a second engineer and a rating rather than of the chief engineer. The last is the most predictive and appears in no dashboard.
What are the acceptable ways for a pilot to end?
Three, and all should be written into the charter. Scale, where the criteria were met under honest conditions and the fleet paper carries observed results rather than projections. Stop, where they were not met for reasons you cannot change — a legitimate outcome that has to be named as legitimate in advance, so ending is a decision rather than an admission. Or extend once, with a named cause and a new fixed date, where the failure traces to something fixable like an over-long checklist or an access model scoped too narrowly. Writing all three down is what prevents the fourth ending, which is continuing indefinitely because none of the others was defined.
Before you install anything
The problem, in one falsifiable sentence
A sponsor and a lead, both named
Baselines, measured this week
Criteria, with numbers
The evaluation date, fixed
All three endings, written down
Pick the Awkward Ship. Use the Messy Data. Set the Date.
A pilot exists to answer a question, and the answer is only worth having if the conditions resemble the fleet. Run it on the vessel you would worry about, with records as they actually are and support as the contract actually provides, starting mid-rotation with a settled crew and ending on a date agreed before anybody logged in. Measure the before so you can prove the after. Then decide — scale, stop, or extend once with a reason — because the only outcome worse than a pilot that fails is one that never ends.
