How many writers will be left? – All rights: fabiusmaximus.com

The world is about to go through a period of unprecedented change and great disruption. Macro-Disrupters” are churning through the workforce, but have not yet reached a Tipping Point (to borrow from Malcolm Gladwell), where we “the big change” becomes visible. A virtuous cycle of better health and education, followed by higher income and smaller families, is erasing many of the differences in the standard of living around the world. This virtuous cycle leads to an end of global population growth by 2050 when the world’s population peaks at 11.5 billion. The population as a whole will no longer grow, but every year more of that population will be over 65 years old, leading to a cascade of disruptive changes to the workforce, the economy and world culture.

Now, a new generation of intelligent “learning” robots are rolling off the assembly line and headed to your workplace. Will your job survive the 21^st century? It’s going to take more than one article to answer that question, which is why this series was created. In each article we look at specific jobs and industries, identifying the technologies that will have the biggest impact. Today’s deep dive will look at creative writing and publishing!

IN BRIEF – PUBLISHING: Traditional print publishing has been in trouble for decades. The arrival of radio and TV news was the start of a long struggle between print and technology. Electronic news could deliver the message faster, but print could deliver in-depth coverage. The Internet broke the time limits of broadcast TV. Digital news could be as lengthy as you liked, and was enriched by multimedia content. Blogging arrived and would eventually grow to 39 million writers in just the US. But were these writers… reporters?

Blog content is sometimes very good (Huffington Post) and sometimes not so good (anything with cats), but bloggers and social media increasingly beat traditional news to the story: earthquakes, disaster sites, political revolutions, live video of controversial arrests, even trend spotting. The Internet also undermined ad revenues by offering advertising that was customized to the reader or to the story. And then Craigslist and competitors offered free classified ads. Not able to compete with rapid market changes, newspapers and magazines have been in steep decline. Since 2000 publishing, in general, lost a third of employees, and newspapers lost two-thirds of revenue.

Book publishing has been equally battered by new technology. In the 1980s, book lovers feared that Barnes and Noble type super bookstores would wipe out small bookstores and the specialty books they sold. Then Amazon.com arrived. Amazon was not the first on-line bookseller, but it quickly became THE dominant bookseller. Amazon also backed disruptive technologies: ebooks, the Kindle e-reader, print on demand and a standard price of $9.99 or less for an ebook (far below the $25-$30 cost for a hardcover book).

Old publishing had strict limits on how many new book titles could be released every year because physical books require investment by the publisher: payments to buy the book, artists to design the cover, promotional materials, and book tours. Publishers would only gain back their investment as books were sold. Of course, some books would lose money. Print on demand and ebooks have virtually no upfront costs, allowing anyone to “publish” a book by simply emailing a file to the e-publisher. The file then sits on a server until purchased, when the ebook is electronically sent to the customer. Or, the file is sent to a robot that prints a single physical book, and then mails it to the purchaser.

Without sunk costs, and with every book sale delivering guaranteed profits, new publishing models are very attractive to authors. Traditional (and hefty) fees charged by Publishers are harder to justify. Technology has made book publishing increasingly challenging, but that same technology has also created a golden age for independent writers, who would never have been given a book to traditional publishers.

Textbooks are a $7 billion market, with very unhappy customers. Unlike books that readers choose, textbooks are assigned by schools and professors. College textbooks typically cost $100 or more, and new editions are issued every few years to prevent students from buying inexpensive used books. School boards want alternatives and are increasingly participating in the “Open Textbook” movement. Just as various “Open Software” groups make software code available for free to the public, Open Textbook allows schools, teachers, students and the general public to freely print (books and ebooks), edit, translate and distribute their content. Use of Open Textbook content can reduce book costs for schools and students by up to 80%. That’s a positive trend for buyers, but not for textbook authors.

NEW TECHNOLOGY: The last generation of robots were industrial. They reduced the cost of publication, distributed e-books, and fueled indie publishing. The new robots are software, rather than physical devices. The key technology is Natural Language Generation and Natural Language Programming. Artificial Intelligence (A.I.) applications can collect information from the Internet or other sources and use it to write a story. Similarly, programs can be told what they need to do, by providing written examples, and feedback on the results.

Programming is out, and learning is in. Even simple tasks can be composed of many even simpler rules. Programming required that these rules be identified, verified, converted into code, tested and then implemented. This process is slow and expensive when performed by humans, but a cheap and quick process when robots learn on their own. Robotic learning is just like human’s learning: provide examples of what needs to be done, rate the work, point out what needs to be corrected, ensure that the right language is used for the right audience, and repeat. New robot writers quickly become experts.

For newspapers and magazines, especially those with a technical focus, today’s generation of robot writers can take over reporter jobs. In the 20th Century, newspapers depended on News Services (ex.: Associated Press and Reuters) to provide stories to fill out their news coverage. Most newspapers use services for national and international news stories and hire reporters to cover local news. The Associated Press (AP) reporters used to write “Quarterly Earnings Reports” for publicly traded firms, but could only produce 300 every quarter. AP turned to the robots of AutomatedInsights, and now quarterly production has risen to 3,750 reports, which AP rates as being as good or better than human written reports. Lower cost, much higher output, and no late reporting because someone was out sick that day.

A competing firm, NarrativeScience produces similar products and recently moved into sports stories. Their A.I. robot can write a story, or even customize stories for specific customers. Custom internet ads created new revenue streams. Could custom reporting, perhaps emphasizing information about local teams, help return newspapers to profitability? Robots can build new bonds with readers by corresponding on forums and social media, or by writing individual emails. This level of reader interaction is not possible, or cost-effective with human writers. Even if robots never produce quite the quality of the very best humans, the speed of robotic writing (which will get faster every year) will transform the position of reporter into something much more interactive, making it impossible for people to perform the function. One last firm of note is Arria in the U.K., which uses technology developed at the University of Aberdeen in Scotland. Expect to see many, many more providers soon.

EMPLOYMENT IMPACT: Writing is often a part-time or unpaid profession: volunteer writers, occasional contributors to professional journals, aspiring writers waiting for their first book sale. Many of these writers are not counted in employment statistics, and are instead reported under their “day job”.  The Bureau of Labor Statistics (BLS) reports that there are over a million workers in Publishing, only 294,000 of which are professional writers and editors, working primarily in magazines, newspapers and textbook departments.

Downward pressure from declining sales and lost advertising revenues have pushed writing towards a robotic solution. The process is already started, and the tools are available. Because these robots are disembodied software, we may not see much visual evidence in the workplace as the robots take over. We can expect that most full-time writing positions will disappear in a decade, with most newspapers and publishers heavily relying on robot writers by 2020. In the peculiar world of Textbook publishing, new 1st editions may still be written by human writers, but the job of publishing 2^nd and later editions should be turned over to robots.

BLS reports another 637,000 workers in Advertising, Marketing, and Public relations. Few of these positions are full-time writers, but many have significant writing responsibilities. Simple robots have been spamming email and writing ads that are stories in disguise. Google and other search engines search for these articles and penalize robotic content. The latest wave of internet ads are cooperative and put an end to the “bot wars”. Advertisers and search engines now work together, presenting “sponsored” or “native” ads, which look like news stories or search results. As news and advertising blur together, tools designed for reporters become increasingly relevant to advertising, marketing, and public relations. Earlier technologies cut telemarketing employment by half, and Natural Language Generation will make advertising a one-to-one relationship that humans cannot provide.

Similarly, corporate managers do a lot of writing and reporting. Before computers, a corporate manager might have spent most of his time producing just one or two reports every month. Today, “spreadsheet” type reports are mostly computer generated, and managers spend time interpret reports and converting the data into insights, usually as a PowerPoint. Academics and Scientists must “Publish or Perish”, producing books and articles to advance in their career. 1.3 million scientists and 1.8 million post-secondary educators do a lot of writing but are rarely counted as writers. Robot writers may make their lives easier, but it’s not likely to have a major impact on employment. However, the number of writers in advertising and marketing will probably shrink by a third, and corporate managers can expect job losses of at least a 10% to 20%.

Robots can write books, but so far they haven’t written any great books. Even if robots become good book authors, the book market will continue to expand until at least 2050, when world population reaches a peak of 11.5 billion. Even after population growth stalls, the book market may continue to expand as populations grow older. Today’s population of 43 million Americans over the age of 65, will grow to 84 million by 2050. A large retired population has more leisure hours for reading.

What about screenwriters? There are only 12,000 members of the screenwriter’s guild, and only 10% of members are listed as “actively employed”. Nonetheless, Hollywood has done much to find a magic formula to simplify the screenwriting process. Blake Snyder authored the book, “Save the Cat”, which claims to have discovered that formula, breaking the script for every movie into 15 “beats”, or events, that must occur in this order in every successful movie. According to Snyder, your script is broken into pages, and each page is one minute of movie time, and you can mark the beats to know exactly where each event will happen. If Snyder broke the code, or if his students write all of Hollywood’s movies (actually, they do) if you read his book you will see that every big American movie made in the last decade follow his beats…. to the minute. Movies are now digitally recorded, increasingly with computer generated graphics. Thanks to Mr. Snyder’s book, robot script writing doesn’t have far to go to remove human creativity from movie making.

SUMMARY: At less than a million jobs, professional writers represent less than 1% of the US labor market. The introduction of Natural Language Generation will quickly tale over a significant number of these jobs, and then automate writing functions for the remaining jobs. Fast and intelligent robot writers will produce hyper-specialized correspondence, articles, and reports, fueling a new age of customer-centric journalism and advertising. Advanced robot writers already create articles for newspapers and professional journals. “New Titles” published by the textbook industry are mostly “new” editions of old books, with little or no new content. Squeezed by budget-constrained school boards and parents, and a growing library of free “Open Textbook” publications, the industry must find a  new business model or turn over post 1^st  edition publishing to a robotic staff. The rapid decline in publishing markets and revenues will drive rapid adoption of Natural Language tools, and by 2020 the robot writer will be a common feature of every publisher.

The impact on book publishing, especially fiction, is harder to predict. More readers, with more leisure time, will increase the size of the book market. At the same time, every potential author will be able to publish a book, setting new records in Publishing, well beyond the 2 million new book titles (US = 300,000) published worldwide today. This growing market is increasingly fragmented: printed books, e-books, self-published books and free books. Expect few millionaire writers, not that the “millionaires club” for authors was ever large.

Natural Language tools, electronic publishing, and other technologies will significantly reduce employment in publishing and advertising, eliminating 30% or more of total employment. While this is significant within publishing, it has little impact on a labor market of 145 million workers. The real impact of robot writers will be in all of the non-writer jobs, that have significant writing responsibilities. Academics, business managers, scientists and other professionals spend a lot of time turning data into presentations and documents. If robot writers make these positions just 10-20% more efficient, millions of positions might be vacated.

The next time you’re at work, take a look around you. Are you one of a large number of people performing exactly the same function? Does your writing follow a template, and have a standard “look and feel”? Your job might be on the front lines of the robot revolution. If you’re a brilliant writer with an established and engaged audience, you don’t need to start clearing out your desk quite yet. But if you spend most of your day writing competent but uninspiring content, it’s time to start thinking about a new profession.