Use crowdsourcing to inject creativity into cutting-edge IT project work

One of the biggest trends in computing is the shift from “normal” programming to data-driven work like machine learning (ML) plus analytics and statistical algorithms. This is where computers’ actions are not directed by a strict set of line-by-line instructions but by an algorithm that responds to input. We are calling this the shift from hard-coding to soft-coding. It is epitomized by artificial intelligence and machine learning, where the algorithm is enhanced (programmed) by data, not hundreds of people coding in pods. For enterprise leaders looking to resource projects in this new world, crowdsourcing offers a unique opportunity to inject creativity into what was previously highly structured project work.

Hard versus soft: why software is hard-coded less as external inputs become more valuable

• Hard-coded software is the product of an engineered and design-led method. In agile projects, developers create and test software in bite-sized pieces and include regular checks to make sure it aligns with a customer’s requirements and is still fit for purpose. Hard-coded software development is a manual task, requiring a developer to encode each operation directly into a program.
• Soft-coded software lets the program take more of its guidance from external input. Machine learning is a good example of this; the algorithm is trained and guided by the data it is fed, and programmers don’t hard-code every permutation.

Incidentally, “hard” here refers to “rigid” rather than “difficult,” otherwise we’d be talking about hard-coded software and even-harder-coded software.

Does soft-coding mean an end to developers, then?

Not quite, but it does mean that the skills are different. Some machine learning models require less “programming” of the type that dictates actions and responses, but the programming that remains can be harder and more intense. In some ways, it’s more like an experiment, especially when the task is very data-science-intensive, such as using a statistical model to identify a specific type of wood from pictures of trees.

Soft-coding also diversifies the type of skills you need; it’s common for problems to require a statistical solution (like the tree example). Statistical solutions combined with programming would require data science skills, and may challenge statistical models when the first answer isn’t the best or most elegant solution—and it might not even be correct. Also, there are many algorithmic methods developers can take to ML, so finding the best version may take several attempts by different teams with different strategies and problem-solving methods, which can be an expensive process.

Competition breeds a diversity of approaches

In the past, competitions were used to drive innovation. A famous example is an 18^th-century competition to find an accurate way of determining longitude at sea; it led to the design of John Harrison’s sea watch, which provided accurate timekeeping that sailors could use in conjunction with a measurement of the sun’s angle to find longitude. Beyond the main solution, other methods competitors used yielded useful solutions, such as lunar maps that allowed calculations at night.

It is this diversity of techniques resulting from competitions that is one of the key benefits of crowdsourcing. Modern crowdsourcing sets a challenge, often without a fixed method, and a prize or set of prizes for the people that take part. Often the final approaches are more diverse if the task isn’t presented with a fixed plan.

How does this work in the (modern) real world?

In a recent example, oncologists from Harvard set about using crowdsourcing to find a way to automatically segment lung tumors for radiation therapy targeting. This test wasn’t about identifying the presence of a tumor; the goal was to locate the precise outline and position of tumors in relation to other organs so that the correct radiotherapy regime could be administered. Radiation doses are triangulated on the tumor by administering the dose from multiple angles. This gives the tumor the correct high dose, lowering the dose to surrounding organs. Tumor delineation is an important part of the therapeutic process; it is time-consuming and requires a specialist.

Harvard used the Topcoder community to undertake the work, and it offered a prize fund of $50,000. The final results performed better than current commercial solutions, and the winning solution performed as well as the expert. The entry algorithms required 15 to 120 seconds per scan, and the human expert required an average of 8 minutes per scan. What was particularly interesting was the variety of approaches the leading teams used, which included convolutional neural networks (CNNs), cluster growth, and random forest algorithms.

The Bottom Line: The wisdom of crowds can help solve complex, cutting-edge problems.

Criticism of crowdsourcing has included both that it can’t be used to do large projects and that the coordination required for multiple intersecting competitions is too expensive. Critics further suggest that the crowd is bad at following up on work with important things like documentation, rewrites, and maintenance. Additionally, a key issue with crowdsourcing has been expecting too much of the crowd. Both sides bear this criticism, but it’s unreasonable for an organization to provide poorly worded briefs and expect a third-party developer to understand and provide a viable solution to a complex problem from the outside without a briefing.

However, the Topcoder/Harvard project shows us that when done well in the right framework, crowdsourcing can inject a great deal of ingenuity into complex projects. These projects do need the right checks and balances; using a platform like Topcoder helps. One of the biggest challenges for many cutting-edge IT projects is that instead of relying on traditional programming, they require designing core AI algorithms and building statistical models—which is particularly challenging with talent in AI and data science being so rare. Seeking highly skilled experts from the crowd can be a shortcut to innovation.