Google Natural Language API and Python

Google’s BERT update has unleashed the incredible potential of natural language processing, or NLP for short, on unsuspecting search queries and the entire body of content that comprises the internet. Despite Google’s claims that BERT shouldn’t drastically change the way we think about content, it’s safe to assume that’s exactly what’s going to happen down the line. Sooner or later, you’ll need to become Google natural language API and Python literate to stay competitive.

Natural language processing, which is at the core of the BERT update, is a powerful tool of understanding. Understanding the intent behind user searches, whether they be typed out strings of letters or phrases uttered. With such an incredible tool at its disposal, Google algorithms can now sequence the query better than ever, understand it, and match it to a website.

Follow on LinkedIn

For users, it’s a dream come true. SERPs are getting ever more accurate, even if you’re using voice searches. And given the sweeping popularity of smartphones, you better believe that’s where most searches are coming from.

For SEO experts, however, the bar just got set much higher than people are used to. Nothing you know is suddenly going to become obsolete. Good SEO practices are still relevant, which is what Google was aiming at when they said your approach to content shouldn’t change.

But in order to stay ahead of the curve, you’ll need to know how to use Python to scour the natural language API and find room for even more content optimization. Right now, it’s not crucial to know that. But we like to be prepared for the future, and the future has “NLP” written all over it.

To that end, we’ve put together this guide to using Google’s natural language API. We’ll show you how to use Google Shell, authenticate API requests, and perform various analyses. This guide is meant to be an introduction to Python and natural language API for beginners.

We encourage you to keep doing research and experimentation on your own using this guide as a basis. Now let’s dive into the world of natural language processing and its practical applications.

Using Cloud Console

We’ll be using Google Cloud Console to test the capabilities of the natural language API. To use Google Cloud Platform, you need a Gmail or Gmail Suite account.

Source: G2

Setting up the Cloud Platform

The first thing you need to do is log in to the Cloud Platform. New Google Cloud users are eligible to receive a free 90-day trial that includes $300 in credit. Give it a try if you haven’t had any prior experience with the Cloud Platform. It’s a good way to test the environment and see if a tool such as this one is right for you.

Once you’re logged in, you can start a new project. Name it whatever you like, but keep track of the project ID that the platform will generate for you. You’ll need it later on.

Now that you have a project to call your own on the Cloud Platform, you can fire up the Cloud Shell. Cloud Shell is a command-line tool that allows you to input instructions and interact with the platform. The Cloud Shell is going to be your main line of communication with the natural language API.

You can find the Cloud Shell button at the top right of the platform screen, next to the help button. Once you click on it, you’ll get a one-time message explaining what Cloud Shell is and that it’s free for all users.

The first command you need to run is the following:

gcloud auth list

You can simply copy and paste the commands into the Cloud Shell. This command (gcloud auth list) will confirm your authentication status. You can’t use Cloud Shell without authenticating your account and credentials. You’ll see the following message:

“Authorize Cloud Shell

gcloud is requesting your credentials to make a GCP API call.

Click to authorize this and future calls that require your credentials.”

Authorize this call to proceed. You can then try the following command to check the active project:

gcloud config list project

The result should be the project ID of the current project you’re working on. If it’s not (for any reason) use:

gcloud config set project

Where is the current project you’re working on.

Enabling Natural Language Processing API and Authenticating Requests

Now that the Cloud Shell is running and is properly authenticated, you need to enable the language processing API. You can do that by entering the following line of code into the shell:

gcloud services enable language.googleapis.com

You’re almost done with the setup! There’s one more thing to do before you get to use Python to explore the depths of natural language API. Python client library uses what’s known as a Service Account to access natural language API. Your Service Account is tied to the project you’re working on, but you still need to create it the first time around.

To authenticate API requests, you first need to set the PROJECT_ID environment variable:

export PROJECT_ID=$(gcloud config get-value core/project)

Then create the service account using this code:

gcloud iam service-accounts create my-nl-sa \

–display-name “my nl service account”

After that’s complete, you’ll need to create and save the credentials that Python will use to log in to your service account. You can do so like this:

gcloud iam service-accounts keys create ~/key.json \

–iam-account my-nl-sa@${PROJECT_ID}.iam.gserviceaccount.com

The Natural Language API Python library needs one more thing, and that’s the GOOGLE_APPLICATION_CREDENTIALS environment variable. One last line of code to set its full path:

export GOOGLE_APPLICATION_CREDENTIALS=~/key.json

That’s it. Your Cloud Shell now has everything it needs to access natural language API.

Source: DataEthics4All

Running Python

Python comes preinstalled in the environment, but if you want to check if it’s there manually, do:

pip3 freeze | grep google-cloud-language

If the Python client library is installed, you’ll see: “google-cloud-language==2.0.0.”

Type in the following command to launch Python:

ipython

From here on out, Cloud Shell will recognize only official Python syntax and rules. If you want to be able to use the gcloud command again, you’ll have to exit the Python environment. Do so by typing “exit” in the Cloud Shell.

Reconnecting

If you lose connection to the Cloud Platform or shut down your PC, you’ll have to authenticate again before using the shell.

You don’t have to repeat the whole process we described above. Most of the actions you’ve taken are one-time commands, so you won’t have to reenter most of the code. But each time you activate the Cloud Shell, you have to run:

gcloud auth list

And then check for the JSON credentials:

export GOOGLE_APPLICATION_CREDENTIALS=~/key.json

If you don’t reenter those two lines we just mentioned, you won’t be able to use Python commands.

Performing Analyses

Your Cloud Shell is finally ready for use. It’s time to begin our exploration of the natural language API and see how such technology can complement the standard SEO toolkit.

We’ll start with sentiment analysis and work our way from there. We’ll give you an executable code sample each time around.

Follow on LinkedIn

You can then modify the code samples further by changing the value of the text variable.

Entity Analysis

Entity analysis is perhaps the most valuable analysis of all. Our sentiment analyses have yielded very little to no significant results, and their uses are limited. Entities, on the other hand, can potentially guide your content in significant ways.

Extracting proper nouns (entities) from a given text can be quite useful, especially if you’re analyzing your competitors’ content. Entity analysis can reveal important aspects of a topic that you haven’t considered so far. Try this code on for size:

from google.cloud import language

def analyze_text_entities(text):

client = language.LanguageServiceClient()

document = language.Document(content=text, type_=language.Document.Type.PLAIN_TEXT)

response = client.analyze_entities(document=document)

for entity in response.entities:

print(“=” * 80)

results = dict(

name=entity.name,

type=entity.type_.name,

salience=f”{entity.salience:.1%}”,

wikipedia_url=entity.metadata.get(“wikipedia_url”, “-“),

mid=entity.metadata.get(“mid”, “-“),

)

for k, v in results.items():

print(f”{k:15}: {v}”)

text = “Johann Sebastian Bach was a German composer of the Baroque period. He is known for various instrumental compositions as well as incredible vocal music.”

analyze_text_entities(text)

Here’s the result you’ll get:

================================================================================

name : Johann Sebastian Bach

type : PERSON

salience : 84.1%

wikipedia_url : https://en.wikipedia.org/wiki/Johann_Sebastian_Bach

mid : /m/03_f0

================================================================================

name : German

type : LOCATION

salience : 10.7%

wikipedia_url : –

mid : –

================================================================================

name : music

type : WORK_OF_ART

salience : 3.0%

wikipedia_url : –

mid : –

================================================================================

name : compositions

type : OTHER

salience : 2.2%

wikipedia_url : –

mid : –

Salience represents the relationship between each entity and the topic of the sentence (or entire article) that you’re analyzing.

Source: Aylien

Syntax Analysis

Syntax analysis performs tokenization of the given text, revealing the structure of your sentences.

The purpose of this section is to show you how Google’s natural language API understands the text. Syntax analysis is the foundation of all other analyses, which is why we’ve included it here.

Tech-savvy SEO experts will be able to train more complex syntax analysis models, but such programmability is beyond the scope of this article. Take a look at how the natural language API sees the text on the most basic level by running the following code sample:

from google.cloud import language

def analyze_text_syntax(text):

client = language.LanguageServiceClient()

document = language.Document(content=text, type_=language.Document.Type.PLAIN_TEXT)

response = client.analyze_syntax(document=document)

fmts = “{:10}: {}”

print(fmts.format(“sentences”, len(response.sentences)))

print(fmts.format(“tokens”, len(response.tokens)))

for token in response.tokens:

print(fmts.format(token.part_of_speech.tag.name, token.text.content))

text = “It was a glorious summer morning and the birds were eagerly chirping away while the crowns of trees were rustled by the winds. Who could have asked for a more serene setting?”

analyze_text_syntax(text)

Here’s the result you’ll get if you run that code:

sentences: 2

tokens: 34

PRON: It

VERB: was

DET: a

ADJ: glorious

NOUN: summer

NOUN: morning

CONJ: and

DET: the

NOUN: birds

VERB: were

ADV: eagerly

VERB: chirping

ADV: away

ADP: while

DET: the

NOUN: crowns

ADP: of

NOUN: trees

VERB: were

VERB: rustled

ADP: by

DET: the

NOUN: winds

PUNCT : .

PRON: Who

VERB: could

VERB: have

VERB: asked

ADP: for

DET: a

ADV: more

ADJ: serene

NOUN: setting

PUNCT: ?

Achieve all your digital marketing goals with us.

Please fill out this short survey to help us get to know you and your business better.

---

PlayMedia is the digital marketing agency that will take you to the top, and help you grow your business through dedication, transparency, and a unique, highly individual approach.

Mladen Maksic CEO
Play Media

Classifying the Content

Google’s natural language API categorizes all of the content into groups that you can find right here. As you can see, there are plenty of subcategories for each category the natural language API recognizes. The API will always try to be as specific as possible when categorizing content.

Categories you’ll see here are the same one recognized by other Google services such as AdWords. Use this code to categorize a short paragraph about Java programming language.

from google.cloud import language

def classify_text(text):

client = language.LanguageServiceClient()

document = language.Document(content=text, type_=language.Document.Type.PLAIN_TEXT)

response = client.classify_text(document=document)

for category in response.categories:

print(“=” * 80)

print(f”category : {category.name}”)

print(f”confidence: {category.confidence:.0%}”)

text = (

“Java is a programming language and computing platform first released by Sun Microsystems in 1995. “

“There are lots of applications and websites that will not work unless you have Java installed, and more are created every day. “

“Java is fast, secure, and reliable. “

“From laptops to datacenters, game consoles to scientific supercomputers, cell phones to the Internet, Java is everywhere!”

)

classify_text(text)

You should see the following output:

================================================================================

category : /Science/Computer Science

confidence: 99%

================================================================================

category : /Computers & Electronics/Programming/Java (Programming Language)

confidence: 94%

To Sum Up

Together, Python and natural language API are powerful tools of savvy SEO experts.

This article covered the basics of performing analyses to discover how Google sees your content and what it thinks about it. Certainly, all the possibilities of the natural language API extended far beyond a single article. But using the sample code we’ve provided, you can begin to scrape crucial data that you’ve never had access to before.

Follow on LinkedIn

It’s up to you to decide how much of the natural language API and Python you’d like to learn. But sooner or later, your optimization strategies will benefit from your newfound knowledge.