Integrating Google APIs and Technologies

In 1997, long before my tenure at Google, I became a member of the Python community in helping to create Yahoo!Mail, one of the most popular web-based email systems in the world. There were only two Python books on the market back then, and neither addressed my developer’s need to learn Python quickly and competently, so I had to resort to the online docs. This absence, and consequently my development of class materials for a Python course, inspired me to write Prentice Hall’s bestselling Core Python Programming over a decade ago. Since then, I’ve used Python to work on all kinds of interesting applications, from web-based e-mail to geolocalized product search engines, social media games, antispam/antivirus e-mail appliances, and most interestingly, software for doctors to help them analyze and assess patients with spinal fractures. (Ask me about osteoporosis!)

Today at Google, my work involves advocating our tools and APIs to the global developer community. Now that I've been part of the Google family for the past 2.5 years, I thought it would be fun to integrate some of our technologies into the book. With the just-published 3rd edition, readers will find revised but also brand new material they can use to build real applications with. Some of the Google technologies I've integrated into Core Python Applications Programming include accessing your Gmail, parsing Google News XML feeds, and a complete chapter on cloud computing with Google App Engine. It’s also the first published book to feature code that utilizes the Google+ API. While the book contains a longer example using that API, I want to show you how easy it is to connect to Google+ using Python right now!

The bulk of the work in connecting to Google+ (and other Google APIs) is done by my fellow colleagues who maintain the Google APIs Client Library for Python, easily downloaded with pip or easy_install as "google-api-python-client." With this library, the most difficult step to connect with your API of choice has basically been reduced to a single line... see the fourth line of this short Python 2.x example:

# plus.py (by Wesley Chun under CC-SA3.0 license)
from apiclient import discovery

API_KEY = YOUR_KEY_FROM_CONSOLE_API_ACCESS_PAGE
service = discovery.build("plus", "v1", developerKey=API_KEY)
feed = service.activities().search(query='android').execute()
for record in feed['items']:
post = ' '.join(record['title'].strip().split())
if post:
print '\nFrom:', record['actor']['displayName']
print 'Post:', post
print 'Date:', record['published']

In that one line of code (italicized above), we use the Google APIs Client Library's apiclient.discovery.build() method, passing in: a) the desired API ("plus" for Google+), b) the version (currently "v1"), and c) the API key you obtained from your project's development console in the "Simple API Access" section. This key gives your project access to APIs that do not need to access user data. Once we have a handle to the service, we can execute generic queries on the available data stream.

In this code snippet, we're simply querying for the latest (public) Google+ posts that are related to Android and displaying them on the command-line (code can be easily repurposed into any mobile or web application). Naturally, you need to go through the OAuth flow if you do want access to authenticated data. Give it a try!

If you like the code, dig into Core Python Applications Programming for a longer, more detailed example. Both scripts can be downloaded at the website (the code is part of Chapter 15), and you can get involved in the conversation on the Google+ page. I'm open to all feedback, suggestions, and fixes. You can find me at +wescpy or @wescpy. Looking forward to meeting you at an upcoming Google or Python event or in one of my public courses!

A new PyCon... and a new book (and a new article)!!

I'm excited about this year's PyCon conference happening this time in the heart of Silicon Valley. There are many firsts, so let's just list a few here (let me know if I'm missing any)!

This is the first PyCon...

• ever held in Silicon Valley (although older Python workshops have been hosted here)
• that had a cap; yes, we "ended" registration at 1500 people
• where we ran out of swag bags; 1800 were ordered... POOF, gone by mid-Saturday
• to have sold out! (even though we capped at 1500; didn't stop it from going over 2000)
• with an attendance near or exceeding 2200 (2257)
• that had to stop accepting sponsorships... at 136!!!
• to feature a physical race (not to be confused with a race condition)

Another exciting announcement is that my first 3rd edition Core Python book will be published and debuting at the conference!! It's called Core Python Applications Programming and based on the second part of the original Core Python Programming book. All of the books' individual home pages are now unified at corepython.com. The books also have a shared Google+ page for you to encircle! They're literally "hot off the presses" as they were overnighted by the printer to the publisher's hotel and brought by hand to the conference! (Amazon's not shipping them for another 10 days after that!)

The new book features upgrades and new stuff added to existing chapters as well as brand new chapters on Django, Google App Engine, and text processing with CSV, JSON, and XML. There is even new material on Twitter and Google+ in case you're feeling more social than when the previous edition was published. Those of you asking for that PowerPoint slideshow generator for the past N years, or perhaps an intro to NoSQL/MongoDB? Yep, they're in there too! Finally, I've added not only Python 3 equivalents to many of the code samples, but I also cover some best practices when porting from 2.x to 3.x.

With all of the updates and new material, I'm hoping that this will be one of the most popular places for intermediate Python programmers to go once they've gotten comfortable with the langauge but want to apply their skills to a variety of topics in Python development today. While the coverage doesn't necessarily go particularly deep, the goal is to give programmers a kickstart with a comprehensive introduction.

To help kickoff the new book, I got to thinking about Python books in general, especially the numerous times that people have either asked me or asked in some online forum: "What's a good Python book?" Unlike Python, there's not one right answer for this question, so as part of this exploration, I came up with 3 different book lists for diverse audiences of readers out there. You can find that article at InformIT.

In the meantime, it's back to the drawing board for me as I prepare to work on the 3rd edition of the main part of Core Python. If you've got ideas or suggestions on updating part 1 or wish to participate in the review process, please contact me now! (@wescpy/+wescpy)

ps. For those interested in brushing up on your Python skills, I'll be offering my popular Intro+Intermediate course this summer near the San Francisco airport. Go to cyberwebconsulting.com for more information!

Writing 2.x & 3.x Compatible Code

While we're at the crossroads transitioning from Python 2 to 3, you may wonder whether it is possible to write code runs without modification under both Python 2 and 3. It seems like a reasonable request, but how would you get started? What breaks the most Python 2 code when executed by a 3.x interpreter?

print vs. print()

If you think like me, you'd say the print statement. That's as good a place to start as any, so let's give it a shot. The tricky part is that in 2.x, it's a statement, thus a keyword or reserved word while in 3.x, it's just a BIF. In other words, because language syntax is involved, you cannot use if statements, and no, Python still doesn't have #ifdef macros!

Let's try just putting parentheses around the arguments to print:

>>> print('Hello World!')

Hello World!

Cool! That works under both Python 2 and Python 3! Are we done? Sorry.

>>> print(10, 20) # Python 2

(10, 20)

You're not going to be as lucky this time as the former is a tuple while in Python 3, you're passing in multiple arguments to print():

>>> print(10, 20) # Python 3

10 20

If you think a bit more, perhaps we can check if print is a keyword. You may recall there is a keyword module which contains a list of keywords. Since print won't be a keyword in 3.x, you may think that it can be as simple as this:

>>> import keyword

>>> 'print' in keyword.kwlist

False

As a smart programmer, you'd probably try it in 2.x expecting a True response. Although you would be correct, you'd still fail for a different reason:

>>> import keyword

>>> if 'print' in keyword.kwlist:

...     from __future__ import print_function

...

File "", line 2

SyntaxError: from __future__ imports must occur at the beginning of the file

One solution which works requires you to use a function that has similar capabilities as print. One of them is sys.stdout.write() while another is distutils.log.warn(). For whatever reason, we decided to use the latter in many of this book's chapters. I suppose sys.stderr.write() will also work, if unbuffered output is your thing.

The "Hello World!" example would then look like this:

# Python 2.x

print 'Hello World!'

# Python 3.x

print('Hello World!')

The following line would work in both versions:

# Python 2.x & 3.x compatible

from distutils.log import warn as printf

printf('Hello World!')

That reminds me of why we didn't use sys.stdout.write()... we would need to add a NEWLINE character at the end of the string to match the behavior:

# Python 2.x & 3.x compatible

import sys

sys.stdout.write('Hello World!\n')

The one real problem isn't this little minor annoyance, but that these functions are no true proxy for print or print() for that matter... they only work when you've come up with a single string representing your output. Anything more complex requires you to put in more effort.

Import your way to a solution

In other situations, life is a bit easier, and you can just import the correct solution. In the code below, we want to import the urlopen() function. In Python 2, it lives in urllib and urllib2 (we'll use the latter), and in Python 3, it's been integrated into urllib.request. Your solution which works for both 2.x and 3.x is neat and simple in this case:

try:

    from urllib2 import urlopen

except ImportError:

    from urllib.request import urlopen

For memory conservation, perhaps you're interested in the iterator (Python 3) version of a well-known built-in like zip(). In Python 2, the iterator version is itertools.izip(). This function is renamed as and replaces zip() in Python 3, and if you insist on this iterator version, your import statement is also fairly straightforward:

try:

    from itertools import izip as zip

except ImportError:

    pass

One example which isn't as elegant looking is the StringIO class. In Python 2, the pure Python version is in the StringIO module, meaning you access it via StringIO.StringIO. There is also a C version for speed, and that's located at cStringIO.StringIO. Depending on your Python installation, you may prefer cStringIO first and fallback to StringIO if cStringIO is not available.

In Python 3, Unicode is the default string type, but if you're doing any kind of networking, it's likely you'll have to manipulate ASCII/bytes strings instead, so instead of StringIO, you'd want io.BytesIO. In order to get what you want, the import is slightly uglier:

try:

    from io import BytesIO as StringIO

except ImportError:

    try:

        from cStringIO import StringIO

    except ImportError:

        from StringIO import StringIO

Putting it all together

If you're lucky, these are all the changes you have to make, and the rest of your code is simpler than the setup at the beginning. If you install the imports above of distutils.log.warn() [as printf()], url*.urlopen(), *.StringIO, and a normal import of xml.etree.ElementTree (2.5 and newer), you can write a very short parser to display the top headline stories from the Google News service with just these roughly eight lines of code:

g = urlopen('http://news.google.com/news?topic=h&output=rss')

f = StringIO(g.read())

g.close()

tree = xml.etree.ElementTree.parse(f)

f.close()

for elmt in tree.getiterator():

    if elmt.tag == 'title' and not \

            elmt.text.startswith('Top Stories'):

        printf('- %s' % elmt.text)

This script runs exactly the same under 2.x and 3.x with no changes to the code whatsoever. Of course, if you're using 2.4 and older, you need to download ElementTree separately.

The code snippets in this subsection come from the "Text Processing" chapter of the book, so take a look at the goognewsrss.py file to see the full version in action.

Some will feel that these changes really start to mess up the elegance of your Python source. After all, readbility counts! If you prefer to keep your code cleaner yet still write code that runs under both versions without changes, take a look at the six package.

six is a compatibility library who's primary role is to provide an interface to keep your application code the same while hiding the complexities described in this appendix subsection from the developer. To find out more about six, read this: http://packages.python.org/six

Regardless whether you use a library like six or choose to roll your own, we hoped to show in this short narrative that it is possible to write code that runs under both 2.x & 3.x. The bottom line is that you may have to sacrifice some of the elegance and simplicity of Python, trading it off for true 2 to 3 portability. I'm sure we'll be revisiting this issue for the next few years until the whole world has completed the transition to the next generation.

Learning Python

Finally here!! Welcome to my new blog dedicated specifically to Python programming. My old blog at LiveJournal is getting a little long in the tooth, plus their spam filtering was lacking... something. However, I'll leave that blogsite up for personal posts, but I'm moving here for Python stuff. In fact, I think I'll "refactor" some of those old posts here from time-to-time.

As you can see, I've named this blog (currently) the same name as my book. This was not done intentionally as some marketing effort to promte the book as I do want to focus on having readers/users understand the core elements of the language. This in turn makes for better Python programmers, thus lowers the stress level in the world a little. Now let's really start the contents of this post, and that means going back to the beginning and learning Python:

If you don't know Python but already code, try the Google Python course first. It is basically the internal 2-day training course scrubbed and externalized for all of you. It jumps in fairly quickly without a lot of explanation. If you learn best in this style, you'll be okay. There are also 7 videos available on the site so you can follow the lectures from both days.

If you're really new to programming, consider taking a beginner course in programming. Sure Python is a great first language to learn coding with, but not all such courses feature it. If you have no time for courses, do the online Python tutorial as well as the SingPath exercises. As far as books go, online-wise you can try Learn Python the Hard Way (book+lessons) or Dive Into Python (book-only).

Since Dive Into Python is written by a co-worker of mine, you can buy a dead-tree version if you wish to support them, or Core Python Programming, to help out someone else you know. :-) The primary difference between these books is that one is a quick dive while the other is a deep dive, as so well described in this Amazon review. Here's another more recent review although it does not shed as much positive light on my colleague's tome. As far as references go, you can support yet a third Googler by buying Python in a Nutshell, or a non-colleague who wrote Python Essential Reference.

If you have children or wish to teach kids how to program, Hello World! Computer Programming for Kids and Other Beginners is a friendly and well-received book written by an engineer and his (then) 8-year old son. It's good to have a kid's perspective as kids (generally) respect other kids who are/were in their same shoes.

The cool thing is that the sky is the limit once you've learned some Python. You can take off in any different direction like Google App Engine, Pyramid, or Django for web or mobile development, SciPy/NumPy for scientific development, SQLAlchemy/SQLObject for database ORMs, Jython for Java development, Win32 for PC development, PyGame for writing games, etc. Testing is something you should always keep in mind, so look into Nose or py.test.

For those of you who already know programming, but want to learn Python as quickly and as in-depth as possible in the shortest amount of time, join me near San Francisco for my upcoming 3-day Python training course running mid-week October 18-20! You need to be proficient programming in another high-level programming language (such as C/C++, Java, PHP, Ruby, etc.). The fee covers for all lectures, labs (3 per day), and everyone gets a copy of my bestseller, Core Python Programming. There is a significant discount to primary/secondary school teachers so ask about that if applicable!

I hope this helps some of you get started! We always welcome new users to the Python community!!