"I Wrote This" Introducing our newest author: A Deep Learning Language Model

By: 
 | 
December 16, 2019
Aug 11, 2022
 | 
Read time: 
5
 min
"I Wrote This" Introducing our newest author: A Deep Learning Language Model

In this data science blog post, we take a deeper look into what data science is, how to implement it in a business environment, and what data science problems the world is currently struggling with (some of us are already starting to solve them). We will also discuss some of the key data science tools and techniques that are currently being used in the data science community.

What is Data Science?

Data science is a field of science that focuses on analyzing and visualizing data. Data science is often used to help companies make decisions about their data. For example, if you are looking to find out what people are doing on your website, you can use data science to figure out what the best way to present your data is. You can also use data science to help companies figure out how to better understand their customers.

There are many different types of data science:

Machine learning: This is the science of understanding how data is organized and presented. Machine learning is a branch of computer science that is primarily focused on making predictions about the behavior of data.

Data mining: This is the science of understanding how data is organized and presented. Data mining is a branch of computer science that is primarily focused on understanding how data is organized and presented.

Data visualization: This is the science of understanding how data is organized and presented. Data visualization is a branch of computer science that is primarily focused on understanding how data is organized and presented.

Data mining and data visualization are not the only types of data science, but they are the most common types of data science.

The first step in implementing data science in your business is to identify the types of data that you want to analyze.

How to Implement Data Science in Your Business

In this blog post, we will discuss how to implement data science in your business. We will also talk about the tools that are currently being used in the data science community to solve these problems.

The first step in implementing data science in your business is to identify the types of data that you want to analyze. You can choose to use data from a variety of sources, like an online survey, a survey that you took, or a survey you did. You can also choose to use data from a variety of sources, like a data warehouse, an online database, or a database that you have created.

Data science is often used to make predictions about the behavior of data. For example, if you are looking to find out how many people have visited your website, you can use data science to figure out what the best way to present your data is. You can also use data science to help companies figure out how to better understand their customers.

The second step in implementing data science in your business is to understand the tools that are currently being used in the data science community to solve these problems.

Method:

The green text above was written by a generative, deep learning neural network by giving it the bolded text at the post beginning as an input prompt. The generated content includes everything following the prompt: the words, punctuation, capitalization, and line breaks.

Try out our interactive webapp to see how the model can be adapted to produce different kinds of text!

Model:

The model, created by OpenAI, is called GPT-2. GPT-2 was trained on text extracted from 8 million websites and is capable of performing well at a variety of Natural Language Understanding (NLU) known as the GLUE tasks - tasks such as text generation (exhibited here), reading comprehension, language translation, and question answering (see this paper for more details). GPT-2 is not the first model to perform well at these tasks, but remarkably, while other models were each trained to perform well on one specific task, GPT-2 was trained task-agnostic and can perform well on a variety of tasks. This post was generated by the second largest version of the model released by OpenAI, which has 774 million parameters.

Analysis:

GPT-2 excels at short text generation, such as sentence completion, but can also perform longer generation as seen here. With longer generation, however, topic coherence declines and some repetition starts to appear, as seen in the second to last paragraph (where the model repeats some of an earlier paragraph), and the first sentence of the three definitions being the same. Regardless, the quality of this output is staggering. On November 5, OpenAI released the largest version of the model that is twice the size of the model used to generate this post. This largest model, containing 1.5 billion parameters, produces even higher quality text, including an article about talking unicorns.

It is important to note that in this text generation task, the model is not attempting to produce facts. So, while the text is convincing of a human author, the model is not (yet) able to produce factual statements with high reliability when generating text in this manner. However, during other NPU tasks, such as Question and Answer, GPT-2 is able to factually answer questions about text passages, though not as well as other models.

If you want to try out the Large (774M) or X-Large (1.5B) models, we have forked the original GPT-2 repo and added a module to facilitate interaction with the model, as well as a notebook that can be imported and run in Google Colab with GPU support at https://github.com/1904labs/gpt-2.

More to come

This post is the first of a four-part series exploring language models, including technical details, ethical implications, and business applications.

Be sure to follow 1904labs on LinkedIn and Twitter to learn when the next article is published!