Advance Praise for Head First Python “Head First Python is a great introduction to not just the Python language, but Python as it’s used in the real world. The book goes beyond the syntax to teach you how to create applications for Android phones, Google’s App Engine, and more.” — David Griffiths, author and Agile coach “Where other books start with theory and progress to examples, Head First Python jumps right in with code and explains the theory as you read along. This is a much more effective learning environment, because it engages the reader to do from the very beginning. It was also just a joy to read. It was fun without being flippant and informative without being condescending. The breadth of examples and explanation covered the majority of what you’ll use in your job every day. I’ll recommend this book to anyone starting out on Python.” — Jeremy Jones, coauthor of Python for Unix and Linux System Administration “Head First Python is a terrific book for getting a grounding in a language that is increasing in relevance day by day.” — Phil Hartley, University of Advancing Technology Download from Wow! eBook
Praise for other Head First books “Kathy and Bert’s Head First Java transforms the printed page into the closest thing to a GUI you’ve ever seen. In a wry, hip manner, the authors make learning Java an engaging ‘what’re they gonna do next?’ experience.” — Warren Keuffel, Software Development Magazine “Beyond the engaging style that drags you forward from know-nothing into exalted Java warrior status, Head First Java covers a huge amount of practical matters that other texts leave as the dreaded ‘exercise for the reader.…’ It’s clever, wry, hip and practical—there aren’t a lot of textbooks that can make that claim and live up to it while also teaching you about object serialization and network launch protocols.” — Dr. Dan Russell, Director of User Sciences and Experience Research IBM Almaden Research Center (and teaches Artificial Intelligence at Stanford University) “It’s fast, irreverent, fun, and engaging. Be careful—you might actually learn something!” — Ken Arnold, former Senior Engineer at Sun Microsystems Coauthor (with James Gosling, creator of Java), The Java Programming Language “I feel like a thousand pounds of books have just been lifted off of my head.” — Ward Cunningham, inventor of the Wiki and founder of the Hillside Group “Just the right tone for the geeked-out, casual-cool guru coder in all of us. The right reference for practical development strategies—gets my brain going without having to slog through a bunch of tired, stale professor-speak.” — Travis Kalanick, founder of Scour and Red Swoosh Member of the MIT TR100 “There are books you buy, books you keep, books you keep on your desk, and thanks to O’Reilly and the Head First crew, there is the penultimate category, Head First books. They’re the ones that are dog-eared, mangled, and carried everywhere. Head First SQL is at the top of my stack. Heck, even the PDF I have for review is tattered and torn.” — Bill Sawyer, ATG Curriculum Manager, Oracle “This book’s admirable clarity, humor and substantial doses of clever make it the sort of book that helps even non-programmers think well about problem-solving.” — Cory Doctorow, co-editor of Boing Boing Author, Down and Out in the Magic Kingdom and Someone Comes to Town, Someone Leaves Town Praise for other Head First books “I received the book yesterday and started to read it…and I couldn’t stop. This is definitely très ‘cool.’ It is fun, but they cover a lot of ground and they are right to the point. I’m really impressed.” — Erich Gamma, IBM Distinguished Engineer, and coauthor of Design Patterns “One of the funniest and smartest books on software design I’ve ever read.” — Aaron LaBerge, VP Technology, ESPN.com “What used to be a long trial and error learning process has now been reduced neatly into an engaging paperback.” — Mike Davidson, CEO, Newsvine, Inc. “Elegant design is at the core of every chapter here, each concept conveyed with equal doses of pragmatism and wit.” — Ken Goldstein, Executive Vice President, Disney Online “I ♥ Head First HTML with CSS & XHTML—it teaches you everything you need to learn in a ‘fun-coated’ format.” — Sally Applin, UI Designer and Artist “Usually when reading through a book or article on design patterns, I’d have to occasionally stick myself in the eye with something just to make sure I was paying attention. Not with this book. Odd as it may sound, this book makes learning about design patterns fun. “While other books on design patterns are saying ‘Bueller…Bueller…Bueller…’ this book is on the float belting out ‘Shake it up, baby!’” — Eric Wuehler “I literally love this book. In fact, I kissed this book in front of my wife.” — Satish Kumar Other related books from O’Reilly Learning Python Programming Python Python in a Nutshell Python Cookbook Python for Unix and Linux System Administration Other books in O’Reilly’s Head First series Head First Algebra Head First Ajax Head First C#, Second Edition Head First Design Patterns Head First EJB Head First Excel Head First 2D Geometry Head First HTML with CSS & XHTML Head First iPhone Development Head First Java Head First JavaScript Head First Object-Oriented Analysis & Design (OOA&D) Head First PHP & MySQL Head First Physics Head First PMP, Second Edition Head First Programming Head First Rails Head First Servlets & JSP, Second Edition Head First Software Development Head First SQL Head First Statistics Head First Web Design Head First WordPress Head First Python Wouldn’t it be dreamy if there were a Python book that didn’t make you wish you were anywhere other than stuck in front of your computer writing code? I guess it’s just a fantasy... Paul Barry Beijing • Cambridge • Farnham • Kln • Sebastopol • Tokyo Head First Python by Paul Barry Copyright © 2011 Paul Barry. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly Media books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/ institutional sales department: (800) 998-9938 or corporate@oreilly.com. Series Creators: Editor: Cover Designer: Production Editor: Proofreader: Indexer: Page Viewers: Kathy Sierra, Bert Bates Brian Sawyer Karen Montgomery Rachel Monaghan Nancy Reinhardt Angela Howard Deirdre, Joseph, Aaron, and Aideen Printing History: November 2010: First Edition. Joseph Deirdre Aideen Aaron The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. The Head First series designations, Head First Python, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and the author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. No athletes were pushed too hard in the making of this book. TM This book uses RepKover™, a durable and flexible lay-flat binding. ISBN: 978-1-449-38267-4 [M] I dedicate this book to all those generous people in the Python community who have helped to make this great little language the first-rate programming technology it is. And to those that made learning Python and its technologies just complex enough that people need a book like this to learn it. the author Author of Head First Python Paul Barry recently worked out that he has been programming for close to a quarter century, a fact that came as a bit of a shock. In that time, Paul has programmed in lots of different programming languages, lived and worked Paul in two countries on two continents, got married, had three kids (well…his wife Deirdre actually had them, but Paul was there), completed a B.Sc. and M.Sc. in Computing, written or cowritten three other books, as well as a bunch of technical articles for Linux Journal (where he’s a Contributing Editor). When Paul first saw Head First HTML with CSS & XHTML, he loved it so much he knew immediately that the Head First approach would be a great way to teach programming. He was only too delighted then, together with David Griffiths, to create Head First Programming in an attempt to prove his hunch correct. Paul’s day job is working as a lecturer at The Institute of Technology, Carlow, in Ireland. As part of the Department of Computing and Networking, Paul gets to spend his day exploring, learning, and teaching programming technologies to his students, including Python. Paul recently completed a post-graduate certificate in Learning and Teaching and was more than a bit relieved to discover that most of what he does conforms to current thirdlevel best practice. viii Table of Contents (Summary) Intro 1 Meet Python: Everyone Loves Lists 2 Sharing Your Code: Modules of Functions 3 Files and Exceptions: Dealing with Errors 4 Persistence: Saving Data to Files 5 Comprehending Data: Work That Data! 6 Custom Data Objects: Bundling Code with Data 7 Web Development: Putting It All Together 8 Mobile App Development: Small Devices 9 Manage Your Data: Handling Input 10 Scaling Your Webapp: Getting Real 11 Dealing with Complexity: Data Wrangling i Leftovers: The Top Ten Things (We Didn’t Cover) table of contents xxiii 1 33 73 105 139 173 213 255 293 351 397 435 Table of Contents (the real thing) Intro Your brain on Python. Here you are trying to learn something, while here your brain is doing you a favor by making sure the learning doesn’t stick. Your brain’s thinking, “Better leave room for more important things, like which wild animals to avoid and whether naked snowboarding is a bad idea.” So how do you trick your brain into thinking that your life depends on knowing Python? Who is this book for? We know what you’re thinking Metacognition Bend your brain into submission Read me The technical review team Acknowledgments xxiv xxv xxvii xxix xxx xxxii xxxiii ix table of contents 1 meet python Everyone loves lists You’re asking one question: “What makes Python different?” The short answer is: lots of things. The longer answers starts by stating that there’s lots that’s familiar, too. Python is a lot like any other general-purpose programming language, with statements, expressions, operators, functions, modules, methods, and classes. All the usual stuff, really. And then there’s the other stuff Python provides that makes the programmer’s life—your life—that little bit easier. You’ll start your tour of Python by learning about lists. But, before getting to that, there’s another important question that needs answering… What’s to like about Python? 2 Install Python 3 3 Use IDLE to help learn Python 4 Work effectively with IDLE 5 Deal with complex data 6 Create simple Python lists 7 Lists are like arrays 9 Add more data to your list 11 Work with your list data 15 For loops work with lists of any size 16 Store lists within lists 18 Check a list for a list 20 Complex data is hard to process 23 Handle many levels of nested lists 24 Don’t repeat code; create a function 28 Create a function in Python 29 Recursion to the rescue! 31 Your Python Toolbox 32 The Holy Grail, 1975, Terry Jones & Terry Gilliam, 91 mins Graham Chapman Michael Palin, John Cleese, Terry Gilliam, Eric Idle & Terry Jones x table of contents 2 sharing your code Modules of functions Reusable code is great, but a shareable module is better. By sharing your code as a Python module, you open up your code to the entire Python community…and it’s always good to share, isn’t it? In this chapter, you’ll learn how to create, install, and distribute your own shareable modules. You’ll then load your module onto Python’s software sharing site on the Web, so that everyone can benefit from your work. Along the way, you’ll pick up a few new tricks relating to Python’s functions, too. It’s too good not to share 34 Turn your function into a module 35 Modules are everywhere 36 Comment your code 37 Prepare your distribution 40 Build your distribution 41 A quick review of your distribution 42 Import a module to use it 43 Python’s modules implement namespaces 45 Register with the PyPI website 47 Upload your code to PyPI 48 Welcome to the PyPI community 49 Control behavior with an extra argument 52 Before your write new code, think BIF 53 Python tries its best to run your code 57 Trace your code 58 Work out what’s wrong 59 Update PyPI with your new code 60 You’ve changed your API 62 nester Use optional arguments 63 Your module supports both APIs 65 Your API is still not right 66 Your module’s reputation is restored 70 nester.py Your Python Toolbox 71 setup.py xi table of contents 3 files and exceptions Dealing with errors It’s simply not enough to process your list data in your code. You need to be able to get your data into your programs with ease, too. It’s no surprise then that Python makes reading data from files easy. Which is great, until you consider what can go wrong when interacting with data external to your programs… and there are lots of things waiting to trip you up! When bad stuff happens, you need a strategy for getting out of trouble, and one such strategy is to deal with any exceptional situations using Python’s exception handling mechanism showcased in this chapter. Data is external to your program 74 It’s all lines of text 75 Take a closer look at the data 77 Know your data 79 Know your methods and ask for help 80 Know your data (better) 82 Two very different approaches 83 Add extra logic 84 Handle exceptions 88 Try first, then recover 89 Identify the code to protect 91 Take a pass on the error 93 What about other errors? 96 Add more error-checking code… 97 …Or add another level of exception handling 98 So, which approach is best? 99 You’re done…except for one small thing 101 Be specific with your exceptions 102 split(beans) Your Python Toolbox 103 xii table of contents persistence 4 Saving data to files It is truly great to be able to process your file-based data. But what happens to your data when you’re done? Of course, it’s best to save your data to a disk file, which allows you to use it again at some later date and time. Taking your memory-based data and storing it to disk is what persistence is all about. Python supports all the usual tools for writing to files and also provides some cool facilities for efficiently storing Python data. Programs produce data 106 Open your file in write mode 110 Files are left open after an exception! 114 Extend try with finally 115 Knowing the type of error is not enough 117 Use with to work with files 120 Default formats are unsuitable for files 124 Why not modify print_lol()? 126 Pickle your data 132 Save with dump and restore with load 133 Generic file I/O with pickle is the way to go! 137 Your Python Toolbox 138 [‘Is this the right room for an argument?’, “No you haven’t!”, ‘When?’, “No you didn’t!”, “You didn’t!”, ‘You did not!’, ‘Ah! (taking out his wallet and paying) Just the five minutes.’, ‘You most certainly did not!’, “Oh no you didn’t!”, “Oh no you didn’t!”, “Oh look, this isn’t an argument!”, “No it isn’t!”, “It’s just contradiction!”, ‘It IS!’, ‘You just contradicted me!’, ‘You DID!’, ‘You did just then!’, ‘(exasperated) Oh, this is futile!!’, ‘Yes it is!’] ['Is this the right room for an argument?', "No you haven't!", 'When?', "No you didn't!", "You didn't!", 'You did not!', 'Ah! (taking out his wallet and paying) Just the five minutes.', 'You most certainly did not!', "Oh no you didn't!", "Oh no you didn't!", "Oh look, this isn't an argument!", "No it isn't!", "It's just contradiction!", 'It IS!', 'You just contradicted me!', 'You DID!', 'You did just then!', '(exasperated) Oh, this is futile!!', 'Yes it is!'] xiii table of contents 5 comprehending data Work that data! Data comes in all shapes and sizes, formats and encodings. To work effectively with your data, you often have to manipulate and transform it into a common format to allow for efficient processing, sorting, and storage. In this chapter, you’ll explore Python goodies that help you work your data up into a sweat, allowing you to achieve data-munging greatness. Coach Kelly needs your help 140 Sort in one of two ways 144 The trouble with time 148 Comprehending lists 155 Iterate to remove duplicates 161 Remove duplicates with sets 166 Your Python Toolbox 172 This chapter’s guaranteed to give you a workout! xiv table of contents 6 custom data objects Bundling code with data It’s important to match your data structure choice to your data. And that choice can make a big difference to the complexity of your code. In Python, although really useful, lists and sets aren’t the only game in town. The Python dictionary lets you organize your data for speedy lookup by associating your data with names, not numbers. And when Python’s built-in data structures don’t quite cut it, the Python class statement lets you define your own. This chapter shows you how. Coach Kelly is back (with a new file format) 174 Use a dictionary to associate data 178 Bundle your code and its data in a class 189 Define a class 190 Use class to define classes 191 The importance of self 192 Every method’s first argument is self 193 Inherit from Python’s built-in list 204 Coach Kelly is impressed 211 Your Python Toolbox 212 The Object Factory xv table of contents 7 web development Putting it all together Sooner or later, you’ll want to share your app with lots of people. You have many options for doing this. Pop your code on PyPI, send out lots of emails, put your code on a CD or USB, or simply install your app manually on the computers of those people who need it. Sounds like a lot of work…not to mention boring. Also, what happens when you produce the next best version of your code? What happens then? How do you manage the update? Let’s face it: it’s such a pain that you’ll think up really creative excuses not to. Luckily, you don’t have to do any of this: just create a webapp instead. And, as this chapter demonstrates, using Python for web development is a breeze. It’s good to share 214 You can put your program on the Web 215 What does your webapp need to do? 218 Design your webapp with MVC 221 Model your data 222 View your interface 226 Control your code 234 CGI lets your web server run programs 235 Display the list of athletes 236 The dreaded 404 error! 242 Create another CGI script 244 Enable CGI tracking to help with errors 248 A small change can make all the difference 250 Your webapp’s a hit! 252 Your Python Toolbox 253 xvi table of contents 8 mobile app development Small devices Putting your data on the Web opens up all types of possibilities. Not only can anyone from anywhere interact with your webapp, but they are increasingly doing so from a collection of diverse computing devices: PCs, laptops, tablets, palmtops, and even mobile phones. And it’s not just humans interacting with your webapp that you have to support and worry about: bots are small programs that can automate web interactions and typically want your data, not your human-friendly HTML. In this chapter, you exploit Python on Coach Kelly’s mobile phone to write an app that interacts with your webapp’s data. The world is getting smaller 256 Coach Kelly is on Android 257 Don’t worry about Python 2 259 Set up your development environment 260 Configure the SDK and emulator 261 Install and configure Android Scripting 262 Add Python to your SL4A installation 263 Test Python on Android 264 Define your app’s requirements 266 The SL4A Android API 274 Select from a list on Android 278 The athlete’s data CGI script 281 The data appears to have changed type 284 JSON can’t handle your custom datatypes 285 Run your app on a real phone 288 Configure AndFTP 289 The coach is thrilled with his app 290 Your Python Toolbox 291 xvii table of contents xviii 9 manage your data Handling input The Web and your phone are not just great ways to display data. They are also great tools to for accepting input from your users. Of course, once your webapp accepts data, it needs to put it somewhere, and the choices you make when deciding what and where this “somewhere” is are often the difference between a webapp that’s easy to grow and extend and one that isn’t. In this chapter, you’ll extend your webapp to accept data from the Web (via a browser or from an Android phone), as well as look at and enhance your back-end data-management services. Your athlete times app has gone national 294 Use a form or dialog to accept input 295 Create an HTML form template 296 The data is delivered to your CGI script 300 Ask for input on your Android phone 304 It’s time to update your server data 308 Avoid race conditions 309 You need a better data storage mechanism 310 Use a database management system 312 Python includes SQLite 313 Exploit Python’s database API 314 The database API as Python code 315 A little database design goes a long way 316 Define your database schema 317 What does the data look like? 318 Transfer the data from your pickle to SQLite 321 What ID is assigned to which athlete? 322 Insert your timing data 323 SQLite data management tools 326 Integrate SQLite with your existing webapp 327 You still need the list of names 332 Get an athlete’s details based on ID 333 You need to amend your Android app, too 342 Update your SQLite-based athlete data 348 The NUAC is over the moon! 349 Your Python Toolbox 350 table of contents 10 scaling your webapp Getting real The Web is a great place to host your app…until things get real. Sooner or later, you’ll hit the jackpot and your webapp will be wildly successful. When that happens, your webapp goes from a handful of hits a day to thousands, possibly ten of thousands, or even more. Will you be ready? Will your web server handle the load? How will you know? What will it cost? Who will pay? Can your data model scale to millions upon millions of data items without slowing to a crawl? Getting a webapp up and running is easy with Python and now, thanks to Google App Engine, scaling a Python webapp is achievable, too. There are whale sightings everywhere 352 The HFWWG needs to automate 353 Build your webapp with Google App Engine 354 Download and install App Engine 355 Make sure App Engine is working 356 App Engine uses the MVC pattern 359 Model your data with App Engine 360 What good is a model without a view? 363 Use templates in App Engine 364 Django’s form validation framework 368 Check your form 369 Controlling your App Engine webapp 370 Restrict input by providing options 376 Meet the “blank screen of death” 378 Process the POST within your webapp 379 Put your data in the datastore 380 Don’t break the “robustness principle” 384 Accept almost any date and time 385 It looks like you’re not quite done yet 388 Sometimes, the tiniest change can make all the difference… 389 Capture your user’s Google ID, too 390 Deploy your webapp to Google’s cloud 391 Your HFWWG webapp is deployed! 394 Your Python Toolbox 395 xix table of contents 11 dealing with complexity Data wrangling It’s great when you can apply Python to a specific domain area. Whether it’s web development, database management, or mobile apps, Python helps you get the job done by not getting in the way of you coding your solution. And then there’s the other types of problems: the ones you can’t categorize or attach to a domain. Problems that are in themselves so unique you have to look at them in a different, highly specific way. Creating bespoke software solutions to these type of problems is an area where Python excels. In this, your final chapter, you’ll stretch your Python skills to the limit and solve problems along the way. What’s a good time goal for the next race? 398 So…what’s the problem? 400 Start with the data 401 Store each time as a dictionary 407 Dissect the prediction code 409 Get input from your user 413 Getting input raises an issue… 414 Search for the closest match 416 The trouble is with time 418 The time-to-seconds-to-time module 419 The trouble is still with time… 422 Port to Android 424 Your Android app is a bunch of dialogs 425 Put your app together… 429 Your app’s a wrap! 431 Your Python Toolbox 432 xx table of contents leftovers i The Top Ten Things (we didn’t cover) You’ve come a long way. But learning about Python is an activity that never stops. The more Python you code, the more you’ll need to learn new ways to do certain things. You’ll need to master new tools and new techniques, too. There’s just not enough room in this book to show you everything you might possibly need to know about Python. So, here’s our list of the top ten things we didn’t cover that you might want to learn more about next. #1: Using a “professional” IDE 436 #2: Coping with scoping 437 #3: Testing 438 #4: Advanced language features 439 #5: Regular expressions 440 #6: More on web frameworks 441 #7: Object relational mappers and NoSQL 442 #8: Programming GUIs 443 #9: Stuff to avoid 444 #10: Other books 445 xxi how to use this book Intro I can’t believe they put that in a Python book. “ISnotwhihsysDecItDiont,hweye answer the put that in baurPnyintghoqnuebsotoiokn?:” xxiii how to use this book Who is this book for? If you can answer “yes” to all of these: 1 Do you already know how to program in another programming language? 2 Do you wish you had the know-how to program Python, add it to your list of tools, and make it do new things? 3 Do you prefer actually doing things and applying the stuff you learn over listening to someone in a lecture rattle on for hours on end? this book is for you. Who should probably back away from this book? If you can answer “yes” to any of these: 1 Do you already know most of what you need to know to program with Python? 2 Are you looking for a reference book to Python, one that covers all the details in excruciating detail? 3 Would you rather have your toenails pulled out by 15 screaming monkeys than learn something new? Do you believe a Python book should cover everything and if it bores the reader to tears in the process then so much the better? this book is not for you. xxiv intro wi[sNe’flolotraecacfnerypootmneamwcaihrtekhcekta,inctgro:eodt.]hitiscbaorodk… the intro We know what you’re thinking “How can this be a serious Python book?” “What’s with all the graphics?” “Can I actually learn it this way?” We know what your brain is thinking TYHouISr bisraiminpotrhtinaknst. Your brain craves novelty. It’s always searching, scanning, waiting for something unusual. It was built that way, and it helps you stay alive. So what does your brain do with all the routine, ordinary, normal things you encounter? Everything it can to stop them from interfering with the brain’s real job—recording things that matter. It doesn’t bother saving the boring things; they never make it past the “this is obviously not important” filter. How does your brain know what’s important? Suppose you’re out for a day hike and a tiger jumps in front of you, what happens inside your head and body? Neurons fire. Emotions crank up. Chemicals surge. And that’s how your brain knows… This must be important! Don’t forget it! But imagine you’re at home, or in a library. It’s a safe, warm, tiger‑free zone. You’re studying. Getting ready for an exam. Or trying to learn some tough technical topic your boss thinks will take a week, ten days at the most. YTosHaurvISinbgirs.ani’nt twhoirntksh Just one problem. Your brain’s trying to do you a big favor. It’s trying to make sure that this obviously non-important content doesn’t clutter up scarce resources. Resources that are better spent storing the really big things. Like tigers. Like the danger of fire. Like how you should never have posted those “party” photos on your Facebook page. And there’s no simple way to tell your brain, “Hey brain, thank you very much, but no matter how dull this book is, and how little I’m registering on the emotional Richter scale right now, I really do want you to keep this stuff around.” Great. Only 450 more dull, dry, boring pages. you are here 4 xxv how to use this book We think of a “Head First” reader as a learner. Syrteoaosukweedahsoraancth’ldtooiftnoemrcsgooiertgtetnaittikht.ieIavten’tsostnecleoxieattnroancnbesoa,ounpmteapeugutrehso.ihbnWiignoe?glokFfganirycos,twtas,nwiyndohtuoeadthyutaoucvuraernthtsiooeynagoadeul.trpBibts,ayrtsacheihnedonoloonmng.aythk, eleeslaaurtnreeisntg SMmuthonoaedmrkeebereoestftiftaoteoncfmvdtiiatvosbherulee(oau.nplHP.atInueomota8at9hdtgh%eerFesipmiawrarpesgoretfoar, vrladeenmsmdaowerlreneniattmirihnnneeigmrnresocpowarrarlililblnanlbneceedtaihuptrarplantetnhowssefto:ewrrgdiscsrteuaaadlposiehnliseki)c,e. alIsytnattdholsmeosyoamrlkveeaelaklpeteearsorttnbohi,linenrgmagtsmhs meruercoltahrhteaend on to the cUtcDooosnon4tenv0e’et%narttsa.backeteoitotnyenovrauelorsnrstesypllafoettsoritao-otlenhsaeearrrnlitoihanuagnsnldtyet.asWpktsehiniirfcgshtahowefnoocaruomllnidaztleeytnodotunsspepta.oTyykelmelel do.stirIroneercarieettltscyeeinntnotstitotsehtnauedtdoroie:eafasdl,esestctritu,muudruseilinnangttgis.nUapgsefiedrrfsiconta-rnpsmeuerearpsdl olaaunrnpt,yguage. cGnseoooxemletvthpreciatnipshngerieoosmb,nlauleeoncmardharsth,nhlaedoepcruartpugwetrhneocts?-optinnhrcoyilvunoosukkiorinmnhgseo,aqadrune.edAsdgtrieeoenenaedspr,ealarytneh.daInnsaecotwottihvkbeinetrioemwwsooltehrtddiavgsta,etiune. nvAdlo,neledsvnsefgoybaorogutthehadacst,t,icidyvueoersuiloyonuffleset,ehxadenycobdhuriaanrilsnnlepenaiurngerdeodsn,tso, mGsibTontNsultaeoentuoholyerattruai,irip—rnatpenwncsrwgslieeatehst.sah’iemYornnskoe,otmeegdocuhunn,utep”resosiktrBoaetbhitiersoonsrraitt.nasbaeanpeiilgltpngfkacymreie—o,gown,vfmneoguieetlttonlyhreheiylen,neeeon“b-etaawgcno.”rrraihdnestnYet‑axyoc.ewmapthueWerdteuirelnrhesirecennegiehenmcrg…,tn’homuhsedcwi?nimnoeon”aege.krk,bxYestnassepnotntoqir’eeuostdwu.wcrnrhititcbhetetahkhrsiardealodatya.it,fnnLybeioyefoo.eporaiuWultaurir’ntsencyeragasain’anvrlaobiboeegzttoifeta.laa“eiybtlIynlnyaooRhtentuuuiaoowdtldken.,rhn!ettY”toihomoostuwehudtega“homrsIthieogrn,cbmem.gtoeaWeesmerlclmettyshehh’ornbswaiemniewtcatgreaaanhwtlrlttehkehthtnioiaononeptygugnli“oetcIeiyua’smodmorfslnouaoomterthltfgshveioeoneeeirsn’loletyabssrhtdodpuelaiimeutcknvphzeIeaezencrtltnyahieoc,ndi,anb’etlgen.t xxvi intro the intro Metacognition: thinking about thinking If you really want to learn, and you want to learn more quickly and more deeply, pay attention to how you pay attention. Think about how you think. Learn how you learn. Most of us did not take courses on metacognition or learning theory when we were growing up. We were expected to learn, but rarely taught to learn. But we assume that if you’re holding this book, you really want to learn how to design user-friendly websites. And you probably don’t want to spend a lot of time. If you want to use what you read in this book, you need to remember what you read. And for that, you’ve got to understand it. To get the most from this book, or any book or learning experience, take responsibility for your brain. Your brain on this content. The trick is to get your brain to see the new material you’re learning as Really Important. Crucial to your well‑being. As important as a tiger. Otherwise, you’re in for a constant battle, with your brain doing its best to keep the new content from sticking. I wonder how I can trick my brain into remembering this stuff... So just how DO you get your brain to treat programming like it was a hungry tiger? There’s the slow, tedious way, or the faster, more effective way. The slow way is about sheer repetition. You obviously know that you are able to learn and remember even the dullest of topics if you keep pounding the same thing into your brain. With enough repetition, your brain says, “This doesn’t feel important to him, but he keeps looking at the same thing over and over and over, so I suppose it must be.” The faster way is to do anything that increases brain activity, especially different types of brain activity. The things on the previous page are a big part of the solution, and they’re all things that have been proven to help your brain work in your favor. For example, studies show that putting words within the pictures they describe (as opposed to somewhere else in the page, like a caption or in the body text) causes your brain to try to makes sense of how the words and picture relate, and this causes more neurons to fire. More neurons firing = more chances for your brain to get that this is something worth paying attention to, and possibly recording. A conversational style helps because people tend to pay more attention when they perceive that they’re in a conversation, since they’re expected to follow along and hold up their end. The amazing thing is, your brain doesn’t necessarily care that the “conversation” is between you and a book! On the other hand, if the writing style is formal and dry, your brain perceives it the same way you experience being lectured to while sitting in a roomful of passive attendees. No need to stay awake. But pictures and conversational style are just the beginning… you are here 4 xxvii how to use this book Here’s what WE did: We used pictures, because your brain is tuned for visuals, not text. As far as your brain’s concerned, a picture really is worth a thousand words. And when text and pictures work together, we embedded the text in the pictures because your brain works more effectively when the text is within the thing the text refers to, as opposed to in a caption or buried in the text somewhere. We used redundancy, saying the same thing in different ways and with different media types, and multiple senses, to increase the chance that the content gets coded into more than one area of your brain. We used concepts and pictures in unexpected ways because your brain is tuned for novelty, and we used pictures and ideas with at least some emotional content, because your brain is tuned to pay attention to the biochemistry of emotions. That which causes you to feel something is more likely to be remembered, even if that feeling is nothing more than a little humor, surprise, or interest. We used a personalized, conversational style, because your brain is tuned to pay more attention when it believes you’re in a conversation than if it thinks you’re passively listening to a presentation. Your brain does this even when you’re reading. We included more than 80 activities, because your brain is tuned to learn and remember more when you do things than when you read about things. And we made the exercises challenging-yet-do-able, because that’s what most people prefer. We used multiple learning styles, because you might prefer step-by-step procedures, while someone else wants to understand the big picture first, and someone else just wants to see an example. But regardless of your own learning preference, everyone benefits from seeing the same content represented in multiple ways. We include content for both sides of your brain, because the more of your brain you engage, the more likely you are to learn and remember, and the longer you can stay focused. Since working one side of the brain often means giving the other side a chance to rest, you can be more productive at learning for a longer period of time. And we included stories and exercises that present more than one point of view, because your brain is tuned to learn more deeply when it’s forced to make evaluations and judgments. We included challenges, with exercises, and by asking questions that don’t always have a straight answer, because your brain is tuned to learn and remember when it has to work at something. Think about it—you can’t get your body in shape just by watching people at the gym. But we did our best to make sure that when you’re working hard, it’s on the right things. That you’re not spending one extra dendrite processing a hard-to-understand example, or parsing difficult, jargon-laden, or overly terse text. We used people. In stories, examples, pictures, etc., because, well, because you’re a person. And your brain pays more attention to people than it does to things. xxviii intro Here’s what YOU can do to bend your brain into submission So, we did our part. The rest is up to you. These tips are a starting point; listen to your brain and figure out what works for you and what doesn’t. Try new things. oCnutyotuhrisreofurtigaenrdatsotri.ck it the intro 1 Slow down. The more you understand, the less you have to memorize. Don’t just read. Stop and think. When the book asks you a question, don’t just skip to the answer. Imagine that someone really is asking the question. The more deeply you force your brain to think, the better chance you have of learning and remembering. 2 Do the exercises. Write your own notes. We put them in, but if we did them for you, that would be like having someone else do your workouts for you. And don’t just look at the exercises. Use a pencil. There’s plenty of evidence that physical activity while learning can increase the learning. 3 Read the “There are No Dumb Questions.” That means all of them. They’re not optional sidebars, they’re part of the core content! Don’t skip them. 4 Make this the last thing you read before bed. Or at least the last challenging thing. Part of the learning (especially the transfer to long-term memory) happens after you put the book down. Your brain needs time on its own, to do more processing. If you put in something new during that processing time, some of what you just learned will be lost. 5 Talk about it. Out loud. Speaking activates a different part of the brain. If you’re trying to understand something, or increase your chance of remembering it later, say it out loud. Better still, try to explain it out loud to someone else. You’ll learn more quickly, and you might uncover ideas you hadn’t known were there when you were reading about it. 6 Drink water. Lots of it. Your brain works best in a nice bath of fluid. Dehydration (which can happen before you ever feel thirsty) decreases cognitive function. 7 Listen to your brain. Pay attention to whether your brain is getting overloaded. If you find yourself starting to skim the surface or forget what you just read, it’s time for a break. Once you go past a certain point, you won’t learn faster by trying to shove more in, and you might even hurt the process. 8 Feel something. Your brain needs to know that this matters. Get involved with the stories. Make up your own captions for the photos. Groaning over a bad joke is still better than feeling nothing at all. 9 Write a lot of code! There’s only one way to learn to program: writing a lot of code. And that’s what you’re going to do throughout this book. Coding is a skill, and the only way to get good at it is to practice. We’re going to give you a lot of practice: every chapter has exercises that pose a problem for you to solve. Don’t just skip over them—a lot of the learning happens when you solve the exercises. We included a solution to each exercise—don’t be afraid to peek at the solution if you get stuck! (It’s easy to get snagged on something small.) But try to solve the problem before you look at the solution. And definitely get it working before you move on to the next part of the book. you are here 4 xxix how to use this book Read Me This is a learning experience, not a reference book. We deliberately stripped out everything that might get in the way of learning whatever it is we’re working on at that point in the book. And the first time through, you need to begin at the beginning, because the book makes assumptions about what you’ve already seen and learned. This book is designed to get you up to speed with Python as quickly as possible. As you need to know stuff, we teach it. So you won’t find long lists of technical material, no tables of Python’s operators, not its operator precedence rules. We don’t cover everything, but we’ve worked really hard to cover the essential material as well as we can, so that you can get Python into your brain quickly and have it stay there. The only assumption we make is that you already know how to program in some other programming language. This book targets Python 3 We use Release 3 of the Python programming language in this book, and we cover how to get and install Python 3 in the first chapter. That said, we don’t completely ignore Release 2, as you’ll discover in Chapters 8 through 11. But trust us, by then you’ll be so happy using Python, you won’t notice that the technologies you’re programming are running Python 2. We put Python to work for you right away. We get you doing useful stuff in Chapter 1 and build from there. There’s no hanging around, because we want you to be productive with Python right away. The activities are NOT optional. The exercises and activities are not add-ons; they’re part of the core content of the book. Some of them are to help with memory, some are for understanding, and some will help you apply what you’ve learned. Don’t skip the exercises. The redundancy is intentional and important. One distinct difference in a Head First book is that we want you to really get it. And we want you to finish the book remembering what you’ve learned. Most reference books don’t have retention and recall as a goal, but this book is about learning, so you’ll see some of the same concepts come up more than once. xxx intro The examples are as lean as possible. Our readers tell us that it’s frustrating to wade through 200 lines of an example looking for the two lines they need to understand. Most examples in this book are shown within the smallest possible context, so that the part you’re trying to learn is clear and simple. Don’t expect all of the examples to be robust, or even complete—they are written specifically for learning, and aren’t always fully functional. We’ve placed a lot of the code examples on the Web so you can copy and paste them as needed. You’ll find them at two locations: http://www.headfirstlabs.com/books/hfpython/ http://python.itcarlow.ie The Brain Power exercises don’t have answers. For some of them, there is no right answer, and for others, part of the learning experience of the Brain Power activities is for you to decide if and when your answers are right. In some of the Brain Power exercises, you will find hints to point you in the right direction. the intro you are here 4 xxxi the review team The technical review team David Griffiths Phil Hartley Jeremy Jones Technical Reviewers: David Griffiths is the author of Head First Rails and the coauthor of Head First Programming. He began programming at age 12, when he saw a documentary on the work of Seymour Papert. At age 15, he wrote an implementation of Papert’s computer language LOGO. After studying Pure Mathematics at University, he began writing code for computers and magazine articles for humans. He’s worked as an agile coach, a developer, and a garage attendant, but not in that order. He can write code in over 10 languages and prose in just one, and when not writing, coding, or coaching, he spends much of his spare time traveling with his lovely wife—and fellow Head First author— Dawn. Phil Hartley has a degree in Computer Science from Edinburgh, Scotland. Having spent more than 30 years in the IT industry with specific expertise in OOP, he is now teaching full time at the University of Advancing Technology in Tempe, AZ. In his spare time, Phil is a raving NFL fanatic Jeremy Jones is coauthor of Python for Unix and Linux System Administration. He has been actively using Python since 2001. He has been a developer, system administrator, quality assurance engineer, and tech support analyst. They all have their rewards and challenges, but his most challenging and rewarding job has been husband and father. xxxii intro the intro Acknowledgments My editor: Brian Sawyer was Head First Python’s editor. When not editing books, Brian likes to run marathons in his spare time. This turns out to be the perfect training for working on another book with me (our second together). O’Reilly and Head First are lucky to have someone of Brian’s caliber working to make this and other books the best they can be. Brian Sawyer The O’Reilly team: Karen Shaner provided administrative support and very capably coordinated the techical review process, responding quickly to my many queries and requests for help. There’s also the back-room gang to thank—the O’Reilly Production Team—who guided this book through its final stages and turned my InDesign files into the beautiful thing you’re holding in your hands right now (or maybe you’re on an iPad, Android tablet, or reading on your PC—that’s cool, too). And thanks to the other Head First authors who, via Twitter, offered cheers, suggestions, and encouragement throughout the entire writing process. You might not think 140 characters make a big difference, but they really do. I am also grateful to Bert Bates who, together with Kathy Sierra, created this series of books with their wonderful Head First Java. At the start of this book, Bert took the time to set the tone with a marathon 90-minute phone call, which stretched my thinking on what I wanted to do to the limit and pushed me to write a better book. Now, some nine months after the phone call, I’m pretty sure I’ve recovered from the mind-bending Bert put me through. Friends and colleagues: My thanks again to Nigel Whyte, Head of Department, Computing and Networking at The Institute of Technology, Carlow, for supporting my involvement in yet another book (especially so soon after the last one). My students (those enrolled on 3rd Year Games Development and 4th Year Software Engineering) have been exposed to this material in various forms over the last 18 months. Their positive reaction to Python and the approach I take with my classes helped inform the structure and eventual content of this book. (And yes, folks, some of this is on your final). Family: My family, Deirdre, Joseph, Aaron, and Aideen had to, once more, bear the grunts and groans, huffs and puffs, and more than a few roars on more than one occasion (although, to be honest, not as often they did with Head First Programming). After the last book, I promised I wouldn’t start another one “for a while.” It turned out “a while” was no more than a few weeks, and I’ll be forever grateful that they didn’t gang up and throw me out of the house for breaking my promise. Without their support, and especially the ongoing love and support of my wife, Deirdre, this book would not have seen the light of day. The without-whom list: My technical review team did an excellent job of keeping me straight and making sure what I covered was spot on. They confirmed when my material was working, challenged me when it wasn’t and not only pointed out when stuff was wrong, but provided suggestions on how to fix it. This is especially true of David Griffiths, my co-conspirator on Head First Programming, whose technical review comments went above and beyond the call of duty. David’s name might not be on the cover of this book, but a lot of his ideas and suggestions grace its pages, and I was thrilled and will forever remain grateful that he approached his role as tech reviewer on Head First Python with such gusto. you are here 4 xxxiii safari books online Safari® Books Online Safari Books Online is an on-demand digital library that lets you easily search over 7,500 technology and creative reference books and videos to find the answers you need quickly. With a subscription, you can read any page and watch any video from our library online. Read books on your cell phone and mobile devices. Access new titles before they are available for print, and get exclusive access to manuscripts in development and post feedback for the authors. Copy and paste code samples, organize your favorites, download chapters, bookmark key sections, create notes, print out pages, and benefit from tons of other time-saving features. O’Reilly Media has uploaded this book to the Safari Books Online service. To have full digital access to this book and others on similar topics from O’Reilly and other publishers, sign up for free at http://my.safaribooksonline.com/?portal=oreilly. xxxiv intro 1 meet python Everyone loves lists Yes, yes...we have lots of Pythons in stock... I’ll just make a quick list. You’re asking one question: “What makes Python different?” The short answer is: lots of things. The longer answers starts by stating that there’s lots that’s familiar, too. Python is a lot like any other general-purpose programming language, with statements, expressions, operators, functions, modules, methods, and classes. All the usual stuff, really. And then there’s the other stuff Python provides that makes the programmer’s life—your life—that little bit easier. You’ll start your tour of Python by learning about lists. But, before getting to that, there’s another important question that needs answering… this is a new chapter 1 python greatness What’s to like about Python? Lots. Rather than tell you, this book’s goal is to show you the greatness that is Python. Yeah... I need something that I can deploy on PCs, Macs, handhelds, phones,the Web, on big servers and small clients...and it has to let me build GUIs quickly and painlessly... OK, yes, yeah, I’m listening... What?!? You’re kidding! Python can do all that? Before diving head first into Python, let’s get a bit of housekeeping out of the way. To work with and execute the Python code in this book, you need a copy of the Python 3 interpreter on your computer. Like a lot of things to do with Python, it’s not difficult to install the interpreter. Assuming, of course, it’s not already there… 2 Chapter 1 meet python Install Python 3 Before you write and run Python code, you need to make sure the Python interpreter is on your computer. In this book, you’ll start out with Release 3 of Python, the very latest (and best) version of the language. A release of Python might already be on your computer. Mac OS X comes with Python 2 preinstalled, as do most versions of Linux (which can also ship with Release 3). Windows, in contrast, doesn’t include any release of Python. Let’s check your computer for Python 3. Open up a command-line prompt and, if you are using Mac OS X or Linux, type: python3 -V On Windows, use this command: “Tv”h,abty’stahne UwPaPy.ERCASE Do this! c:\Python31\python.exe -V If Python 3 is missing from your computer, download a copy for your favorite OS UUrvseePisrnPusgliEtotRnshCiaenApptSehEaer“iPnvgy”tohnon screen. File Edit Window Help WhichPython? $ python3 -V Python 3.1.2 $ from the www.python.org website. WyUiinnoPittutPoeharEotrpuRehrteCetttAaPehkSyreeEt.nh“ovn”, $ python3 Python 3.1.2 (r312:79360M, Mar 24 2010, 01:33:18) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "help", "copyright", "credits" or "license" for more info. >>> >>> quit() $ Uctrohesmeteumtirnahntneedtrqopturiyoteo(teu)exrritOaSnd prompt. When you install Python 3, you also get IDLE, Python’s simple—yet surprisingly useful— integrated development environment. IDLE includes a color syntax-highlighting editor, a debugger, the Python Shell, and a complete copy of Python 3’s online documentation set. Let’s take a quick look at IDLE. you are here 4 3 idle hands Use IDLE to help learn Python IDLE lets you write code in its full-featured code editor as well as experiment with code at the Python Shell. You’ll use the code editor later in this book but, when learning Python, IDLE’s shell really rocks, because it lets you try out new Python code as you go. When you first start IDLE, you are presented with the “triple chevron” prompt (>>>) at which you enter code. The shell takes your code statement and immediately executes it for you, displaying any results produced on screen. IDLE knows all about Python syntax and offers “completion hints” that pop up when you use a built-in function like print(). Python programmers generally refer to built-in functions as BIFs. The print() BIF displays messages to standard output (usually the screen). Eatnttehrey>o>ur> cpordoempt. itlUnaodnnegdlinuketaeligametoistito,hnwberlhioniccsCtkh-se,aubsdPaes.yet{dhaonndu}ses See results immediately. IDLE uses colored syntax to highlight your code. By default, built-in functions are purple, strings are green, and language keywords (like if) are orange. Any results produced are in blue. If you hate these color choices, don’t worry; you can easily change them by adjusting IDLE’s preferences. IDLE also knows all about Python’s indentation syntax, which requires code blocks be indented. When you start with Python, this can be hard to get used to, but IDLE keeps you straight by automatically indenting as needed. 4 Chapter 1 IDLE knows Python’s syntax and helps you conform to the Python indentation rules. Work effectively with IDLE IDLE has lots of features, but you need to know about only a few of them to get going. TAB completion Start to type in some code, and then press the TAB key. IDLE will offer suggestions to help you complete your statement. TTtcAoyopmBseemea“atpnIrDdt”hLcaeEonm>d’sp>tll>eihsttpeinrooonfmpt suggestions. meet python mnTlmioteyhtatinslcebtoiysmdtimphofuouftwlceoehrorI.ekD.n(ItAtLthnEoidmsn,liuogyygohoelktuyss,r.)lsooi,tno’bksuta Recall code statements Press Alt-P to recall the previous code statement entered into IDLE or press Alt-N to move to the next code statement Alt-P for Previous (assuming there is one). Both key combinations can be used to cycle rapidly through all of the code you’ve entered into Alt-N for Next IDLE, re-executing any code statements as needed. Edit recalled code Once you recall your code statement, you can edit it and move around the statement using the arrow keys. It’s UacanMsleeassict,y’sionuC’wrtehrilco-hnP and Ctrl-N. possible to edit any statement that you’ve previously entered, even code statements that span multiple lines. Adjust IDLE’s preferences IDLE’s preferences dialog lets you adjust its default behavior to your tastes. There are four tabs of settings to tweak. You can control font and tab behavior, the colors used to syntax highlight, the behavior of certain key- TtwoeyaokurIDhLeaErt’s content. combinations, and IDLE’s start-up settings. So, if shocking pink strings is really your thing, IDLE gives you the power to change how your code looks on screen. you are here 4 5 dealing with data Deal with complex data Any program of any worth that you create has to work with data. Sometimes, the data is simple and straightforward—easy to work with. Other times, the data you have to work with is complex in its structure and meaning, forcing you to work hard to make sense of it all, let alone write code to process it. To tame complexity, you can often arrange your data as a list: there’s the list of customers, your friend’s list, the shopping list, and your to-do list (to name a few). Arranging data in lists is so common that Python makes it easy for you to create and process lists in code. Let’s look at some complex data before learning how to create and process list data with Python. Aomrhogviaginehilzybeudff I’ve been making lists of movie data for years and would love to be able to process it on my laptop... Ta hloetreosfurdeatisa listed here. The Holy Grail, 1975, Terry Jones & Terry Gilliam, 91 mins Graham Chapman Michael Palin, John Cleese , Terry Gilliam, Eric Idle & Terry Jones The Life of Brian, 1979, Terry Jones, 94 mins Graham Chapman Michael Palin, John Cleese , Terry Gilliam, Eric Idle & Terry Jones The Meaning of Life, 1983, Terry Jones, 107 mins The six Monty Python cast members The Holy Grail, 1975, Terry Jones & Terry Gilliam, 91 mins Graham Chapman Michael Palin, John Cleese , Terry Gilliam, Eric Idle & Terry Jones The Life of Brian, 1979, Terry Jones, 94 mins Graham Chapman Michael Palin, John Cleese , Terry Gilliam, Eric Idle & Terry Jones The Meaning of Life, 1983, Terry Jones, 107 mins The six Monty Python cast members The Holy Grail, 1975, Terry Jones & Terry Gilliam, 91 mins Graham Chapman Michael Palin, John Cleese , Terry Gilliam, Eric Idle & Terry Jones The Life of Brian, 1979, Terry Jones, 94 mins Graham Chapman Michael Palin, John Cleese , Terry Gilliam, Eric Idle & Terry Jones The Meaning of Life, 1983, Terry Jones, 107 mins The six Monty Python cast members The Holy Grail, 1975, Terry Jones & Terry Gilliam, 91 mins Graham Chapman Michael Palin, John Cleese, Terry Gilliam, Eric Idle & Terry Jones The Life of Brian, 1979, Terry Jones, 94 mins Graham Chapman Michael Palin, John Cleese, Terry Gilliam, Eric Idle & Terry Jones The Meaning of Life, 1983, Terry Jones, 107 mins The six Monty Python cast members Graham Chapman, Michael Palin, John Cleese, Terry Gilliam, Eric Idle & Terry Jones This data is complex, too. On first glance, this collection of data does indeed look quite complex. However, the data appears to conform to some sort of structure: there’s a line for a list of basic movie facts, then another line for the lead actor(s), followed by a third line listing the movie’s supporting actors. This looks like a structure you can work with… 6 Chapter 1 Create simple Python lists Let’s start with the following simple list of movie titles and work up from there: The Holy Grail The Life of Brian The Meaning of Life AMoshnotrytPlyistthoofn some movies Here’s the same list written in a way that Python understands: meet python movies = ["The Holy Grail", "The Life of Brian", "The Meaning of Life"] To turn the human-friendly list into a Python-friendly one, follow this fourstep process: 1 Convert each of the names into strings by surrounding the data with quotes. 2 Separate each of the list items from the next with a comma. 3 Surround the list of items with opening and closing square brackets. 4 Assign the list to an identifier (movies in the preceding code) using the assignment operator (=). It’s perfectly OK to put your list creation code all on one line, assuming, of course, that you have room: movies = ["The Holy Grail", "The Life of Brian", "The Meaning of Life"] This works, too. you are here 4 7 not my type Hang on a second! Aren’t you forgetting something? Don’t you need to declare type information for your list? No, because Python’s variable identifiers don’t have a type. Many other programming languages insist that every identifier used in code has type information declared for it. Not so with Python: identifiers are simply names that refer to a data object of some type. Think of Python’s list as a high-level collection. The type of the data items is not important to the list. It’s OK to state that your movies list is a “collection of strings,” but Python doesn’t need to be told this. All Python needs to know is that you need a list, you’ve given it a name, and the list has some data items in it. 8 Chapter 1 meet python Lists are like arrays When you create a list in Python, the interpreter creates an array-like data structure in memory to hold your data, with your data items stacked from the bottom up. Like array technology in other programming languages, the first slot in the stack is numbered 0, the second is numbered 1, the third is numbered 2, and so on: “Tmhoisvieiss”yoliustr in code. Item #0 Item #1 Item #2 movies = ["The Holy Grail", "The Life of Brian", "The Meaning of Life"] This is list in your “movies” memory. "The Meaning of Life" "The Life of Brian" "The Holy Grail" Access list data using the square bracket notation As with arrays, you can access the data item in a list slot using the standard square bracket offset notation: 2 1 aniEnsusamotccehhrieaidtclaeisOtdtaFwhFiatitSsehEmaTit. 0 Python starts counting from zero. print(movies[1]) The Life of Brian Udasetathiteem“proinnts(c)r”eeBnI.F to display a rNeoquseusrtperdisedahtearea, prpeeaallrys…otnhescreen. Let’s use IDLE to learn a bit about how lists work. you are here 4 9 idle session Lists in Python might look like arrays, but they are much more than that: they are full-blown Python collection objects. This means that lists come with ready-to-use functionality in the form of list methods. Let’s get to know some of Python’s list methods. Open up IDLE and follow along with the code entered at the >>> prompt. You should see exactly the same output as shown here. Start by defining a list of names, which you then display on screen using the print() BIF. Then, use the len() BIF to work out how many data items are in the list, before accessing and displaying the value of the second data item: >>> cast = ["Cleese", 'Palin', 'Jones', "Idle"] >>> print(cast) ['Cleese', 'Palin', 'Jones', 'Idle'] >>> print(len(cast)) 4 >>> print(cast[1]) It’s the OK to results inovfokaenoathBeIrFBoInF. Palin With your list created, you can use list methods to add a single data item to the end of your list (using the append() method), remove data from the end of your list (with the pop() method), and add a collection of data items to the end of your list (thanks to the extend() method): >>> cast.append("Gilliam") >>> print(cast) Mcoemtmhoonds“.a”rdeoitnvnooktedatuiosnin.g the ['Cleese', 'Palin', 'Jones', 'Idle', 'Gilliam'] >>> cast.pop() 'Gilliam' >>> print(cast) ['Cleese', 'Palin', 'Jones', 'Idle'] Istu’rsraonunodthederblyists:quitaermesbsreapcakreattse. d by commas, >>> cast.extend(["Gilliam", "Chapman"]) >>> print(cast) ['Cleese', 'Palin', 'Jones', 'Idle', 'Gilliam', 'Chapman'] Finally, find and remove a specific data item from your list (with the remove() method) and then add a data item before a specific slot location (using the insert() method): >>> cast.remove("Chapman") >>> print(cast) ['Cleese', 'Palin', 'Jones', 'Idle', 'Gilliam'] >>> cast.insert(0, "Chapman") >>> print(cast) ['Chapman', 'Cleese', 'Palin', 'Jones', 'Idle', 'Gilliam'] FtAhlfyetinecgrasCatilrloctfuhsMa!to, nwtey end up with Python’s 10 Chapter 1 Add more data to your list With your list of movie names created, now you need to add more of the movie buff ’s complex data to it. You have a choice here: I think I’ll use the appropriate list methods to add the extra data I need. meet python With something this small, I’m gonna simply re-create my list from scratch. Either strategy works. Which works best for you depends on what you are trying to do. Let’s recall what the movie buff ’s data looks like: Arenpurmesbeenrting the year is next. The Holy Grail, 1975, Terry Jones & Terry Gilliam, 91 mins Graham Chapman Michael Palin, John Cleese, Terry Gilliam, Eric Idle & Terry Jones The Life of Brian, 1979, Terry Jones, 94 mins Graham Chapman Michael Palin, John Cleese, Terry Gilliam, Eric Idle & Terry Jones The Meaning of Life, 1983, Terry Jones, 107 mins The six Monty Python cast members Graham Chapman, Michael Palin, John Cleese, Terry Gilliam, Eric Idle & Terry Jones The next piece of data you need to add to your list is a number (which represents the year the movie was released), and it must be inserted after each movie name. Let’s do that and see what happens. you are here 4 11 mixed type What?!? There’s no way you can mix data of different types in lists, is there? Surely this is madness? No, not madness, just the way Python works. Python lists can contain data of mixed type. It’s perfectly OK to mix strings with numbers within the same Python list. In fact, you can mix more than just strings and numbers; you can store data of any type in a single list, if you like. Recall that a Python list is a high-level collection, designed from the get-go to store a collection of “related things.” What type those things have is of little interest to the list, because the list exists merely to provide the mechanism to store data in list form. So, if you really need to store data of mixed type in a list, Python won’t stop you. 12 Chapter 1 meet python Let’s take a bit of time to try to work out which strategy to use when adding data to your list in this case. Given the following list-creation code: movies = ["The Holy Grail", "The Life of Brian", "The Meaning of Life"] 1 Work out the Python code required to insert the numeric year data into the preceding list, changing the list so that it ends up looking like this: ["The Holy Grail", 1975, "The Life of Brian", 1979, "The Meaning of Life", 1983] Write your insertion code here. 2 Now write the Python code required to re-create the list with the data you need all in one go: Write your re-creation code here. In this case, which of these two methods do you think is best? (Circle your choice). 1 or 2 you are here 4 13 additional data § Let’s take a bit of time to try and work out which strategy to use when adding data to your list in this case. Given the following list-creation code: movies = ["The Holy Grail", "The Life of Brian", "The Meaning of Life"] 1 You were to work out the Python code required to insert the numeric year data into the preceding list: IBnsEeFrtORtEhetfhiersstecyoenadr list item. BInEseFrOtRtEhetsheecofnoduryteharlist item. movies.insert(1, 1975) movies.insert(3, 1979) movies.append(1983) swtlADeihoscftiraodttkngeidynrrinogoitutwnoohssguee,ectrstoftowni.trshyishdeoterueerimnhatsaateotviroehtdntiorowoingth,hthettaen?hkee Tthheenenadppoefndtthheelilsats.t year to 2 You were also to write the Python code required to recreate the list with the data you need all in one go: movies = ["The Holy Grail", 1975, Aidsesingtnifailelry.oWurhadtatwaastoprtehvieou“smlyovtiehse”re is replaced. "The Life of Brian", 1979, "The Meaning of Life", 1983] In this case, which of these two methods do you think is best? (You were to circle your choice.) 1 or 2 Yoltiepsrstti,ciolkminykeethcheotoruhednis…t.2itnAhgslasettoeo,mitsds,hotfe.orhere’sabenstomtaelrl 14 Chapter 1 meet python Work with your list data You often need to iterate over your list and perform some action on each item as you go along. Of course, it is always possible to do something like this, which works but does not scale: miDtoeevmfiesisn.ewiathlisttheanndampeospuolaftetwitos fav_movies = ["The Holy Grail", "The Life of Brian"] Deoaniscphtlahiyendtsicvhriedeuevanal.luleistofitem print(fav_movies[0]) This is the list-processing print(fav_movies[1]) code. This code works as expected, making the data from the list appear on screen. However, if the code is later amended to add another favorite movie to the list, the list-processing code stops working as expected, because the list-processing code does not mention the third item. Big deal: all you need to do is add another print() statement, right? Yes, adding one extra print() statement works for one extra movie, but what if you need to add another hundred favorite movies? The scale of the problem defeats you, because adding all those extra print() statements becomes such a chore that you would rather find an excuse not to have to do. It’s time to iterate Processing every list item is such a common requirement that Python makes it especially convenient, with the built-in for loop. Consider this code, which is a rewrite of the previous Uotisvnshceederrie“vvetfiadnohluureaae”slliotsiytftooe,uiemtdageicsoorhpn.altayeing code to use a for loop: fav_movies = ["The Holy Grail", for each_flick in fav_movies: print(each_flick) jDusetfianseyaoulisdtidanbdefpoorpeu.late "The Life of Brian"] This is the code, using list-processing a for loop. it Using a for loop scales and works with any size list. you are here 4 15 list processing For loops work with lists of any size Python’s for loop exists to process lists and other iterations in Python. Lists are the most common iterated data structure in Python, and when you need to iterate a list, it’s best to use for: Tioctnfhoademritcgekahesetetybewlseiodofoterophndreteai“nfstftdihoeaerrr”.t for The keyword “in” separates the target identifier from your list. in target identifer list ptlAirhstoecconesltsaosamnirnet“g:a”conoffddoyleilno.oduwriscalyitsoteu-sr : list-processing code TMuhnUedSelTirstt-bheperionfcdoeresnsltioneogdp.code The list-processing code is referred to by Python programmers as the suite. The target identifier is like any other name in your code. As your list is iterated over, the target identifier is assigned each of the data values in your list, in turn. This means that each time the loop code executes, the target identifier refers to a different data value. The loop keeps iterating until it exhausts all of your list’s data, no matter how big or small your list is. An alternative to using for is to code the iteration with a while loop. Consider these two snippets of Python code, which perform the same action: “yWwstotohhuaietcehnhemaypvironeleoufqytouuoriasrmewecaso“torwuyironohyntuil,i”ean”bg,out identifier. count = 0 while count < len(movies): print(movies[count]) count = count+1 These while and for statements do the same thing. for each_item in movies: print(each_item) iwPWnofyhroterrhnimeosnaytoaiuinbotonueu”srtepf“rotferhtoeyreo”r“,ust.tahtee 16 Chapter 1 meet python Q: So…when iterating over a list, I should always use for instead of while? A: Yes, unless you have a really good reason to use (or need the extra control of) a while loop. The for loop takes care of working from the start of your list and continuing to the end. It’s next to impossible to get stung by an off-by-one error when you use for. This is not the case with while. Q: So, lists aren’t really like arrays then, because they do so much more? A: Well…they are in that you can access individual data items in your list with the standard square bracket notation, but—as you’ve seen—Python’s lists can do so much more. At Head First Labs, we like to think of lists as “arrays on steroids.” Q: And they work this way only in Python 3, right? A: No. There are certain enhancements to lists that were added in Python 3, but release 2 of Python has lists, too. All of what you’ve learned about lists so far will work with lists in Releases 2 and 3 of Python. Q: Why are we using Python 3? What’s wrong with Python 2, anyway? Lots of programmers seem to be using it. A: Lots of programmers are using Python 2, but the future of Python development lies with Release 3. Of course, moving the entire Python community to Python 3 won’t happen overnight, so there’s an awful lot of projects that will continue to run on Release 2 for the foreseeable future. Despite 2’s dominance at the moment, at Head First Labs we think the new bits in 3 are well worth the added investment in learning about them now. Don’t worry: if you know 2, Python 3 is easy. Q: Seeing as Python’s lists shrink and grow as needed, they must not support bounds-checking, right? A: Well, lists are dynamic, in that they shrink and grow, but they are not magic, in that they cannot access a data item that does not exist. If you try to access a nonexistent data item, Python responds with an IndexError, which means “out of bounds.” Q: What’s with all the strange references to Monty Python? A: Ah, you spotted that, eh? It turns out that the creator of Python, Guido van Rossum, was reading the scripts of the Monty Python TV shows while designing his new programming language. When Guido needed a name for his new language, he chose “Python” as a bit of a joke (or so the legend goes). Q: Do I need to know Monty Python in order to understand the examples? A: No, but as they say in the official Python documentation: “it helps if you do.” But don’t worry: you’ll survive, even if you’ve never heard of Monty Python. Q: I notice that some of your strings are surrounded with double quotes and others with single quotes. What’s the difference? A: There isn’t any. Python lets you use either to create a string. The only rule is that if you start a string with one of the quotes, then you have to end it with the same quote; you can’t mix’n’match. As you may have seen, IDLE uses single quotes when displaying strings within the shell. Q: What if I need to embed a double quote in a string? A: You have two choices: either escape the double quote like this: \”, or surround your string with single quotes. Q: Can I use any characters to name my identifiers? A: No. Like most other programming languages, Python has some rules that must be adhered to when creating names. Names can start with a letter character or an underscore, then include any number of letter characters, numbers, and/or underscores in the rest of the name. Strange characters (such as %$£) are not allowed and you’ll obviously want to use names that have meaning within the context of your code. Names like members, the_ time , and people are much better than m, t, and p, aren’t they? Q: Yes, good naming practice is always important. But what about case sensitivity? A: Yes, Python is the “sensitive type,” in that Python code is case sensitive. This means that msg and MSG are two different names, so be careful. Python (and IDLE) will help with the problems that can occur as a result of this. For instance, you can use an identifier in your code only if it has been given a value; unassigned identifiers cause a runtime error. This means that if you type mgs when you meant msg, you’ll find out pretty quickly when Python complains about your code having a NameError. you are here 4 17 lists within lists Store lists within lists As you’ve seen, lists can hold data of mixed type. But it gets even better than that: lists can hold collections of anything, including other lists. Simply embed the inner list within the enclosing list as needed. Looking closely at the movie buff ’s data, it is possible to determine a structure which looks much like a list of lists: There’s only actor listed there could one lead here, but be more. There’s a list of movie facts… …which itself contains a list of lead actors… …scuwopnhptiocarhitniistnsgaelalficsttoorfs. The Holy Grail, 1975, Terry Jones & Terry Gilliam, 91 mins Graham Chapman Michael Palin, John Cleese, Terry Gilliam, Eric Idle & Terry Jones In Python, you can turn this real list of data into code with little or no effort. All you need to remember is that every list is a collection of items separated from each other with commas and surrounded with square brackets. And, of course, any list item can itself be another list: Tfihrests,toaurtteorflistthe movies = [ "The Holy Grail", 1975, "Terry Jones & Terry Gilliam", 91, The lists end of is here. all the ["Graham Chapman", ["Michael Palin", "John Cleese", "Terry Gilliam", "Eric Idle", "Terry Jones"]]] Tsehceonsdt,arintneorf “movies[4]” the list: The start inner list: “omfovtihees[t4h]ir[d1], ”inner So, a list within a list is possible, as is a list within a list within a list (as this example code demonstrates). In fact, it’s possible to nest lists within lists to most any level with Python. And you can manipulate every list with its own list methods and access it with the square bracket notation: aTorlpeshemoinseibnmleogboteskrhqsrutaeaehrlaeicttlbtortlseaihncewgkreeeoitrnasde,rs…es. uotnhttrhileeeyreoumust print(movies[4][1][3]) A list within a list within a list 18 Chapter 1 Eric Idle cEarnic’tispotshsiibslydebeepliydlnee.s§ted, so he meet python Creating a list that contains another list is straightforward. But what happens when you try to process a list that contains another list (or lists) using the for loop from earlier in this chapter? Let’s use IDLE to work out what happens. Begin by creating the list of the movie data for “The Holy Grail” in memory, display it on screen, and then process the list with your for loop: >>> movies = ["The Holy Grail", 1975, "Terry Jones & Terry Gilliam", 91, ["Graham Chapman", ["Michael Palin", "John Cleese", "Terry Gilliam", "Eric Idle", "Terry Jones"]]] >>> print(movies) ['The Holy Grail', 1975, 'Terry Jones & Terry Gilliam', 91, ['Graham Chapman', ['Michael Palin', 'John Cleese', 'Terry Gilliam', 'Eric Idle', 'Terry Jones']]] >>> for each_item in movies: print(each_item) Tahleistlishtaswibteheinn a list within created in memory. The Holy Grail 1975 Terry Jones & Terry Gilliam The “for” loop prints each item of the outer loop ONLY. 91 ['Graham Chapman', ['Michael Palin', 'John Cleese', 'Terry Gilliam', 'Eric Idle', 'Terry Jones']] The inner list within the inner list is printed “as-is.” Your for loop is working OK. I think the trouble is that you haven’t told it what to do with any inner lists that it finds, so it just prints everything, right? Yes, that’s correct: the loop code isn’t complete. At the moment, the code within the loop simply prints each list item, and when it finds a list at a slot, it simply displays the entire list on screen. After all, the inner list is just another list item as far as the outer enclosing list is concerned. What’s we need here is some mechanism to spot that an item in a list is in fact another list and take the appropriate action. That sounds a little tricky. But can Python help? you are here 4 19 looking for lists Check a list for a list Each time you process an item in your list, you need to check to see if the item is another list. If the item is a list, you need to process the nested list before processing the next item in your outer list. Deciding what to do when in Python follows the familiar if... else... pattern: Tionfhdeitckaheteyewds oetcrhidseio“snitfac”rotde. if some condition holds Aconcdolitonion(:)tefsotll.ows your : Tchonisdcitoidoen ehxoledcsut(ei.se.i,fitt’sheTRUE). else: This does code NOT ehxoeldcu(tie.es.,ifit’tshFeAcLonSdEit).ion the "true" suite Look! Another colon. the "false" suite Naroetein: dbeontthedsu.ites No surprises here, as the if statement in Python works pretty much as expected. But what condition do you need to check? You need a way to determine if the item currently being processed is a list. Luckily, Python ships with a BIF that can help here: isinstance(). What’s cool about the isinstance() BIF is that it lets you check if a specific identifier holds data of a specific type: aCsrseiganteitatsohoarntidliestntaifnider. Ask if “names” is a list (it is). Assign a number to an identifier. Ask list i(fit“insunm’t_).names” is a 20 Chapter 1 Let’s use the IDLE shell to learn a little about how isinstance() works: >>> names = ['Michael', 'Terry'] >>> isinstance(names, list) True >>> num_names = len(names) >>> isinstance(num_names, list) False Rheerfee.rIntothaisPcyatsheo,nttheypteype is “list”. meet python Here’s a copy of the current list-processing code. Your task is to rewrite this code using an if statement and the isinstance() BIF to process a list that displays another list. for each_item in movies: print(each_item) Write your new code here. Q: Are there many of these BIFs in Python? A: Yes. At the last count, there were over 70 BIFs in Python 3. Q: Over 70! How am I to remember that many, let alone find out what they all are? A: You don’t have to worry about remembering. Let Python do it for you. Q: How? A: At the Python or IDLE shell, type dir(__builtins__) to see a list of the built-in stuff that comes with Python (that’s two leading and trailing underscore characters, by the way). The shell spits out a big list. Try it. All those lowercase words are BIFs. To find out what any BIF does—like input(), for example—type help(input) at the shell for a description of the BIFs function. Q: Why so many BIFs? A: Why not? Because Python comes with lots of built-in functionality, it can mean less code for you to write. This Python philosophy is known as “batteries included”: there’s enough included with Python to let you do most things well, without having to rely on code from third parties to get going. As well as lots of BIFs, you’ll find that Python’s standard library is rich and packed with features waiting to be exploited by you. you are here 4 21 list the list Here’s a copy of the current list-processing code. Your task was to rewrite this code using an if statement and the isinstance() BIF to process a list that displays another list. for each_item in movies: print(each_item) Plisrtocaesssbtefhoere“m. ovies” You need to check if the current item is a list. for each_item in movies: if isinstance(each_item, list): Tindeheeendtsiinfanieenrre.lwootparget If it is a list, use a noth er for nested_item in each_item: “for” loop to nested list. proces s the print(nested_item) else: iDndidenytoautimonanraiggehtt?o get your print(each_item) If the of the isn’t a current item enclosing list list, display it on screen. Let’s use IDLE to see if this code makes a difference to the output displayed on screen: >>> for each_item in movies: if isinstance(each_item, list): for nested_item in each_item: print(nested_item) else: print(each_item) The Holy Grail 1975 Terry Jones & Terry Gilliam Tmthhuiacsthi’…ss tnahoeltirtebt’esleinabgneopttrthoeecrre,snsbeeusdttepndrootlpisebtrylhye. re 91 Graham Chapman [‘Michael Palin’, ‘John Cleese’, ‘Terry Gilliam’, ‘Eric Idle’, ‘Terry Jones’] 22 Chapter 1 Complex data is hard to process The movie buff ’s data is complex. Let’s take another look at a subset of the data and your Python code that processes it. The data The outer, enclosing list The Holy Grail, 1975, Terry Jones & Terry Gilliam, 91 mins Graham Chapman Michael Palin, John Cleese, Terry Gilliam, Eric Idle & Terry Jones meet python An inner, nested list Another inner (inner), nested list Your code Process the outer, enclosing list. nPersotceedsslitsth.e inner, for each_item in movies: if isinstance(each_item, list): for nested_item in each_item: print(nested_item) else: print(each_item) Yeah.. that’s almost working...it’s just a pity about that list of supporting actors... Can you spot the problem with your Python code as it is currently written? What do you think needs to happen to your code to allow it to process the movie buff’s data correctly? you are here 4 23 nested lists Handle many levels of nested lists The data and your code are not in sync. The movie buff ’s data is a list that contains a nested list that itself contains a nested list. The trouble is that your code knows only how to process a list nested inside an enclosing list. The solution, of course, is to add more code to handle the additionally nested list. By looking at the existing code, it’s easy to spot the code you need to repeat: This code processes a nested list. for each_item in movies: if isinstance(each_item, list): for nested_item in each_item: print(nested_item) else: print(each_item) Ticthoedereantelioxootnksoflikyeour this. trHoeeprgeeoa’.ts ewdhecroedethneeeds for each_item in movies: if isinstance(each_item, list): for nested_item in each_item: Ncoodtee,: e in acthhis if isinstance(nested_item, list): for deeper_item in nested_item: “ if” nee ds an print(deeper_item) associa ted else: “else”. print(nested_item) else: print(each_item) casTratneloahlpetteldahercme“eeredpsneetttaeahtpraeeegnrde“d_tpcirtioinidedntmetern(”ot).d”ifucieers 24 Chapter 1 meet python Let’s use IDLE once more to test this latest iteration of your code: >>> for each_item in movies: if isinstance(each_item, list): for nested_item in each_item: if isinstance(nested_item, list): for deeper_item in nested_item: print(deeper_item) else: print(nested_item) else: print(each_item) The Holy Grail 1975 Terry Jones & Terry Gilliam 91 Graham Chapman Michael Palin John Cleese Terry Gilliam Eric Idle Terry Jones It works! see all of on screen. This your time, you list data Pliirnstosicdienesssiadane deaneecnpleolsystinengdesltliisesttd. I just love that...in fact, I love it so much I’ve decided to add another list to my data. I want to include the other movies each supporting actor has starred in. If I add the data, can you change your code to print this data, too? That’s more list data and more Python code. The data has to be embedded as another nested list within the already deeply nested list of supporting actors. That’s possible to do, even though it makes your head hurt just to think about a list of lists of lists of lists! Amending your code is just a matter of adding another for loop and an if statement. That doesn’t sound like too much trouble, does it? you are here 4 25 avoid complexity I think I’d rather have a root canal than change that code again. Adding another nested loop is a huge pain. Your data is getting more complex (that mind-bending list of lists of lists of lists) and, as a consequence, your code is getting overly complex, too (that brain-exploding for loop inside a for loop inside a for loop). And overly complex code is rarely a good thing… 26 Chapter 1 meet python Wouldn’t it be dreamy if there were an efficient way to process lists, preferably using a technique that resulted in less code, not more? But I know it’s just a fantasy... you are here 4 27 reduce, reuse, recycle Don’t repeat code; create a function Take a look at the code that you’ve created so far, which (in an effort to save you from having your brain explode) has already been amended to process yet another nested list. Notice anything? This same code is as this essentially code… the for each_item in movies: if isinstance(each_item, list): for nested_item in each_item: …which is the same essentially as this code… if isinstance(nested_item, list): for deeper_item in nested_item: if isinstance(deeper_item, list): for deepest_item in deeper_item: print(deepest_item) else: print(deeper_item) else: print(nested_item) else: print(each_item) Tdfhiofeufrreer’sstenanctoeetmamemnuotcnhsg, these either! t…dthiwhfisahftieccrohmednuiestc.hntohtan This code is beginning to get a little scary… Your code now contains a lot of repeated code. It’s also a mess to look at, even though it works with the movie buff ’s amended data. All that nesting of for loops is hard to read, and it’s even harder to ensure that the else suites are associated with the correct if statement. There has to be a better way…but what to do? When code repeats in this way, most programmers look for a way to take the general pattern of the code and turn it into a reusable function. And Python programmers think this way, too. Creating a reusable function lets you invoke the function as needed, as opposed to cutting and pasting existing code. So, let’s turn the repeating code into a function. 28 Chapter 1 meet python Create a function in Python A function in Python is a named suite of code, which can also take an optional list of arguments if required. You define a Python function using the def statement, providing a name for your function and specifying either an empty or populated argument list within parentheses. The standard form looks something like this: Tionfhtertokhdeeuycwfeuosnrctdthi“eodnne.afm”e AburtgutmheentpalirsetnstahreeseospatrioenNalO, T. saActnlaodcrsoitnilnogdonfipc(aa:yr)toeefunsrotlthloehwseiss the functions code suite. def ( ): function name argument(s) TMthhUeeSfdTuenfbctesitoianntd’seemnctoedendte. under function code suite What does your function need to do? Your function needs to take a list and process each item in the list. If it finds a nested list within the first list, the function needs to repeat. It can do this by invoking itself on the nested list. In other words, the function needs to recur— that is, invoke itself from within the funtion code suite. Let’s call the function that you’ll create print_lol(). It takes one argument: a list to display on screen. Grab your pencil and complete the code below to provide the required functionality: def print_lol(the_list): for if else: you are here 4 29 recursive function You were to call the function that you’ll create print_lol(). It takes one argument: a list to display on screen. You were to grab your pencil and complete the code to provide the required functionality: def print_lol(the_list): Plisrtocwesitshthae“fporro”vidl oeodp. for each_item in the_list: if isinstance(each_item, list): fiIsfunittctsheieolfnit.aemlisbte, iinngvopkreoctehsesed else: print_lol(each_item) print(each_item) If the a list, item being display the pitreomcesosnedscIrSeNen’T. Let’s use IDLE one final time to test your new function. Will it work as well as your earlier code? >>> def print_lol(the_list): for each_item in the_list: if isinstance(each_item, list): print_lol(each_item) else: print(each_item) Define the function. >>> print_lol(movies) The Holy Grail 1975 Terry Jones & Terry Gilliam 91 Graham Chapman Michael Palin John Cleese Terry Gilliam Eric Idle Terry Jones Invoke the function. tpIthreowdeouarcrkelssi,eErtoXcooA!dCTeT.heLYrectuhserisvaemfeunrcetsuioltns as 30 Chapter 1 Recursion to the rescue! The use of a recursive function has allowed you to reduce 14 lines of messy, hard-to-understand, brain-hurting code into a six-line function. Unlike the earlier code that needs to be amended to support additional nested lists (should the movie buff require them), the recursive function does not need to change to process any depth of nested lists properly. Python 3 defaults its recursion limit to 1,000, which is a lot of lists of lists of lists of lists…and this limit can be changed should you ever need even more depth than that. Ah, yes, that’s terrific! I can now relax, knowing that your code can process my movie data. I really should’ve done this years ago... meet python What a great start! By taking advantage of functions and recursion, you’ve solved the code complexity problems that had crept into your earlier list-processing code. By creating print_lol(), you’ve produced a reusable chunk of code that can be put to use in many places in your (and others) programs. You’re well on your way to putting Python to work! you are here 4 31 CHAPTER 1 CHAPTER 1 python toolbox Your Python Toolbox You’ve got Chapter 1 under your belt and you’ve added some key Python goodies to your toolbox. Python Lingo • “BIF” - a built-in function. • “Suite” is indented atobilnodckicaotfePgyrtohuopinngc.ode, which •rwgeo“itfBinehagrtmrqtionuesgirtcikteelosyveitnarhcnyeldtudhfpiaenrdcgot”dyut-ochutaa’litlvwenPlayeyye. tdohftono comes get IDLE Notes • The IDLE your code as shell lets you you write it. experiment with • Adjust IDLE’s way you work. preferences to suit the N•useRexetAml(etbm-ubPteruf:soewr hCPetrnrevlwiioofurskyioanung’drweuitsoehn the shell, Alt-N for a Mac). Run Python 3 from the command line or from within IDLE. Identifiers are names that refer to data objects. The identifiers have no “type,” but the data objects that they refer to do. print() BIF displays a message on screen. A list is a collection of data, separated by commas and surrounded by square brackets. Lists are like arrays on steroids. Lists can be used with BIFs, but also support a bunch of list methods. Lists can hold any data, and the data can be of mixed type. Lists can also hold other lists. Lists shrink and grow as needed. All of the memory used by your data is managed by Python for you. Python uses indentation to group statements together. len() BIF provides a length of some data object or count the number of items in a collection, such as a list. The for loop lets you iterate a list and is often more convenient to use that an equivalent while loop. The if... else... statement lets you make decisions in your code. isinstance() BIF checks whether an identifier refers to a data object of some specified type. Use def to define a custom function. 32 Chapter 1 2 sharing your code Modules of functions I’d love to share...but how am I supposed to function without a module? Reusable code is great, but a shareable module is better. By sharing your code as a Python module, you open up your code to the entire Python community…and it’s always good to share, isn’t it? In this chapter, you’ll learn how to create, install, and distribute your own shareable modules. You’ll then load your module onto Python’s software sharing site on the Web, so that everyone can benefit from your work. Along the way, you’ll pick up a few new tricks relating to Python’s functions, too. this is a new chapter 33 let’s share It’s too good not to share You’ve been showing your function to other programmers, and they like what they see. You should make your function shareable, so that everyone can use it. Yes, a function this good should be shared with the world. Python provides a set of technologies that make this easy for you, which includes modules and the distribution utilities: Modules let you organize your code for optimal sharing. The distribution utilities let you share your modules with the world. Let’s turn your function into a module, then use the distribution utilities to share your module with the wider Python programming community. 34 Chapter 2 Turn your function into a module A module is simply a text file that contains Python code. The main requirement is that the name of the file needs to end in .py: the Python extension. To turn your function into a module, save your code into an appropriately named file: Your code from Chapter 1 def print_lol(the_list): for each_item in the_list: if isinstance(each_item, list): print_lol(each_item) else: print(each_item) sharing your code Let’s call this file “nester.py”. Q: What’s the best Python editor? A: The answer to that question really depends on who you ask. However, you can, of course, use any text editor to create and save your function’s code in a text file. Something as simple as NotePad on Windows works fine for this, as does a full-featured editor such as TextMate on Mac OS X. And there’s also full-fledged IDEs such as Eclipse on Linux, as well as the classic vi and emacs editors. And, as you already know, Python comes with IDLE, which also includes a built-in code editor. It might not be as capable as those other “real” editors, but IDLE is installed with Python and is essentially guaranteed to be available. For lots of jobs, IDLE’s edit window is all the editor you’ll ever need when working with your Python code. Of course, there are other IDEs for Python, too. Check out WingIDE for one that specifically targets Python developers. Do this! Go ahead and create a text file called nester.py that contains your function code from the end of Chapter 1. you are here 4 35 modules repository Modules are everywhere As might be expected, you’ll find Python modules in lots of places. I’m preloaded with lots of modules in the Python Standard Library... and they are already on your computer. If the Standard Library doesn’t do it for you, why not try the Web? I hear PyPI is where third-party Python modules hang out. “PpyieP-Ipiies.”pronounced The Python Package Index (or PyPI for short) provides a centralized repository for third-party Python modules on the Internet. When you are ready, you’ll use PyPI to publish your module and make your code available for use by others. And your module is ready, but for one important addition. What do you think is missing from your module? 36 Chapter 2 Geek Bits If you are already familiar with Perl’s CPAN repository, you can think of PyPI as the Python equivalent. sharing your code Comment your code It’s always a good idea to include comments with your code. As your plan to share your module with the world, well-written comments help to document your work. In Python, a common commenting technique is to use a triple quote for multiple-line comments. When you use a triple quote without assigning it to a variable, everything between the triple quotes is considered a comment: Start with a triple quote… Hello! I’m a big string who just happens to be a Python comment, too. Nice, eh? """This is the standard way to include a multiple-line comment in your code.""" …tarnipdleenqduowteit. h a cPoumtmyeonutr module here. Here is your module code (which is saved in the file nester.py). In the spaces provided, use your pencil to compose two comments: the first to describe the module and the second to describe the function. def print_lol(the_list): Add for yaoucromfmunecnttion here. for each_item in the_list: if isinstance(each_item, list) print_lol(each_item) else: print(each_item) you are here 4 37 request for comments Here is your module code (which is saved in the file nester.py). In the spaces provided, you were asked to use your pencil to compose two comments: the first to describe the module and the second to describe the function. “““This is the “nester.py" module, and it provides one function called print_lol() which prints lists that may or may not include nested lists.””” Drineidcmleuydmoebuetrhetotriple def print_lol(the_list): “““This function takes a positional argument called “the_list", which is any quotes? Python list (of, possibly, nested lists). Each data item in the provided list is (recursively) printed to the screen on its own line.””” for each_item in the_list: if isinstance(each_item, list): print_lol(each_item) else: print(each_item) Taacdhteduriaenlgacrsooedmeneohcecorhmea;mnygeoensut’srt.eo the just Q: How do I know where the Python modules are on my computer? A: Ask IDLE. Type import sys; sys.path (all on one line) into the IDLE prompt to see the list of locations that your Python interpreter searches for modules. Q: Hang on a second. I can use “;” to put more than one line of code on the same line in my Python programs? A: Yes, you can. However, I don’t recommend that you do so. Better to give each Python statement its own line; it makes your code much easier for you (and others) to read. Q: Does it matter where I put my nester.py module? A: For now, no. Just be sure to put it somewhere where you can find it later. In a while, you’ll install your module into your local copy of Python, so that the interpreter can find it without you having to remember when you actually put it. Q: So comments are like a funny- looking string surrounded by quotes? A: Yes. When a triple-quoted string is not assigned to a variable, it’s treated like a comment. The comments in your code are surrounded by three double quotes, but you could have used single quotes, too. Q: Is there any other way to add a comment to Python code? A: Yes. If you put a “#” symbol anywhere on a line, everything from that point to the end of the current line is a comment (unless the “#” appears within a triple quote, in which case it’s part of that comment). A lot of Python programmers use the “#” symbol to quickly switch on and off a single line of code when testing new functionality. 38 Chapter 2 sharing your code Now that you’ve added your comments and created a module, let’s test that your code is still working properly. Rather than typing your function’s code into IDLE’s prompt, bring the nester.py file into IDLE’s edit window, and then press F5 to run the module’s code: Nccooomtloemretcnhotadsteadtr.hee Nothing appears to happen, other than the Python shell “restarting” and an empty prompt appearing: >>> ================================ RESTART ================================ >>> >>> What’s happened is that the Python interpreter has reset and the code in your module has executed. The code defines the function but, other than that, does little else. The interpreter is patiently waiting for you to do something with your newly defined function, so let’s create a list of lists and invoke the function on it: >>> movies = [ "The Holy Grail", 1975, "Terry Jones & Terry Gilliam", 91, DfreofmineChtahpetleirst1.of movies facts ["Graham Chapman", ["Michael Palin", "John Cleese", "Terry Gilliam", "Eric Idle", "Terry Jones"]]] >>> print_lol(movies) The Holy Grail Invoke the function on the list. 1975 Terry Jones & Terry Gilliam 91 Graham Chapman Michael Palin John Cleese Cfinuonotclh.teiYoolniustrascoeofxdpleiestcctsoenidts .indTuisehpselatydoeadta on screen. Terry Gilliam Eric Idle Terry Jones you are here 4 39 distribution plan Prepare your distribution In order to share your newly created module, you need to prepare a distribution. This is the Python name given to the collection of files that together allow you to build, package, and distribute your module. Once a distribution exists, you can install your module into your local copy of Python, as well as upload your module to PyPI to share with the world. Follow along with the process described on these two pages to create a distribution for your module. 1 Begin by creating a folder for your module. With the folder created, copy your nester.py module file into the folder. To keep things simple, let’s call the folder nester: The “nester.py” module file. nester Do this! Follow along with each of the steps described on these pages. By the time you reach the end, your module will have transformed into a Python distribution. “Tdniehrseetcentreo”wryfly)o.lcdreerat(eodr 2 Create a file called “setup.py” in your new folder. This file contains metadata about your distribution. Edit this file by adding the following code: “Ifsdmeriptsotoumrrpit”bPufytttuhinheocnotniu’ostnilities. from distutils.core import setup setup( name = 'nester', aTserhtgeuusmpeefanurtnecnttaihmoene’ss. version py_modules author = '1.0.0', = ['nester'], = 'hfpython', Associate metadata function’s your module’s awrigtuhmtehnetss.etup author_email = 'hfpython@headfirstlabs.com', url = 'http://www.headfirstlabs.com', description = 'A simple printer of nested lists', ) wytTLvoiahluahlleurbeiberssemsemueHdasoteeridfaeawfdudletaietrthFshee;ainrtst. 40 Chapter 2 sharing your code Build your distribution You now have a folder with two files in it: your module’s code in nester.py and metadata about your module in setup.py. Now, it’s time to build your distribution. 3 Build a distribution file. wrNeitophtlaec“:cei:f\“Ppyyyottuhhoaonrn3e3”1u\sipninygtthhWeosinne.decxoowem”s.m, ands The distribution utilities include all of the smarts required to build a distribution. Open a terminal window within your nester folder and type a single command: python3 setup.py sdist. Enter the command at the prompt. File Edit Window Help Build $ python3 setup.py sdist running sdist running check warning: sdist: manifest template 'MANIFEST.in' does not exist Asacctporacpnetoefaulatliserricosmmtnioeinonosgnsfastcogyhrfeoeesuern, distribution. warning: sdist: standard file not found: should have README writing manifest file 'MANIFEST' creating nester-1.0.0 making hard links in nester-1.0.0... hard linking nester.py -> nester-1.0.0 hard linking setup.py -> nester-1.0.0 creating dist Creating tar archive removing 'nester-1.0.0' (and everything under it) $ 4 Install your distribution into your local copy of Python. Staying in the terminal, type this command: sudo python3 setup.py install. icaAsntopsnntapofteataluilrhrsametmroiinonebgnssusctoanrfhgceheeesno,f your distribution. File Edit Window Help Install $ python3 setup.py install running install running build running build_py creating build creating build/lib copying nester.py -> build/lib running install_lib copying build/lib/nester.py -> /Library/Frameworks/Python. framework/Versions/3.1/lib/python3.1/site-packages byte-compiling /Library/Frameworks/Python.framework/Versions/3.1/ lib/python3.1/site-packages/nester.py to nester.pyc running install_egg_info Writing /Library/Frameworks/Python.framework/Versions/3.1/lib/ python3.1/site-packages/nester-1.0.0-py3.1.egg-info Your distribution is ready. you are here 4 41 ready for distribution A quick review of your distribution Thanks to Python’s distribution utilities, your module has been transformed into a distribution and installed into your local copy of Python. You started with a single function, which you entered into a file called nester.py, creating a module. You then created a folder called nester to house your module. The addition of a file called setup.py to your folder allowed you to build and install your distribution, which has resulted in a number of additional files and two new folders appearing within your nester folder. These files and folders are all created for you by the distribution utilities. Before Setup After Setup nester Your code this file. is in nester.py setup.py Your metadata is in this file. nester MANIFEST build yiAnoutlirshtidsiosfftirlefib.iluetsioinn is Here are your new folders. lib Your code is in this file. 42 Chapter 2 Your code this file. is in Your metadata is in this file. nester.py dist This is your distribution package. nester-1.0.0.tar.gz nester.py nester.pyc setup.py A “compiled” of your code this file. version is in sharing your code Import a module to use it Now that your module is built, packaged as a distribution, and installed, let’s see what’s involved in using it. To use a module, simply import it into your programs or import it into the IDLE shell: Use the keyword Python “import”… import nester …oafndyopurromvioddeutleh.e name Niwnohcteluned: enyaotmuhiendgo“n.yp’otyu”rneemexdtodetnuosleio.n The import statement tells Python to include the nester.py module in your program. From that point on, you can use the module’s functions as if they were entered directly into your program, right? Well…that’s what you might expect. Let’s check out the validity of your assumption. Write a small program that imports your newly created module, defines a small list called “cast,” and then uses the function provided by your module to display the contents of the list on screen. Use the following list data (all strings): Palin, Cleese, Idle, Jones, Gilliam, and Chapman. Open your program in IDLE’s edit window, and then press F5 to execute your code. Describe what happens in the space below: you are here 4 43 idle error You were to write a small program that imports your newly created module, defines a small list called “cast,” and then uses the function provided by your module to display the contents of the list on screen. You were to use the following list data (all strings): Palin, Cleese, Idle, Jones, Gilliam, and Chapman. Ilntino’esthapirnsogimgtrpoaloem.dtThifrhfeeiecr-uel’ts here. import nester cast = ['Palin’, 'Cleese’, 'Idle’, 'Jones’, 'Gilliam’, 'Chapman’] print_lol(cast) Open your program in IDLE’s edit window, and then press F5 to execute your code. Describe what happens in the space below: But it didn’t work! IDLE gives an error, and the program does not run! With your program in the IDLE edit window, pressing F5 (or choosing Run Module from the Run menu) does indeed cause problems: Your program does not appear to have executed and an error message is reported: >>> ================================ RESTART ================================ >>> Traceback (most recent call last): File "/Users/barryp/HeadFirstPython/chapter2/try_nester.py", line 4, in print_lol(cast) NameError: name 'print_lol' is not defined >>> faWouNitnahdm!y!e!oEurrrporro…girtalmooikns IlDikLe Ey,ouprrefsusinncgtiFon5 causes can’t be 44 Chapter 2 Python’s modules implement namespaces All code in Python is associated with a namespace. Code in your main Python program (and within IDLE’s shell) is associated with a namespace called __main__. When you put your code into its own module, Python automatically creates a namespace with the same name as your module. So, the code in your module is associated with a namespace called nester. sharing your code That’s a front of after it. dtohueblweourndde“rmsacionr”eainnd I guess namespaces are like family names? If someone is looking for Chris, we need to know if it’s Chris Murray or Chris Larkin, right? The family name helps to qualify what we mean, as do namespace names in Python. Yes, namespace names are like family names. When you want to refer to some function from a module namespace other than the one you are currently in, you need to qualify the invocation of the function with the module’s namespace name. So, instead of invoking the function as print_lol(cast) you need to qualify the name as nester.print_lol(cast). That way, the Python interpreter knows where to look. The format for namespace qualification is: the module’s name, followed by a period, and then the function name. Tidheentmifodieuslethneamnea,mwehspicahce. nester.print_lol(cast) tTnhoheremlifasutl,nwcttoitiophnr“oicscaetsstsh.”enprinovvoidkeedd as as Anapmeeriofdrosmeptahreatfeusntcthieonmnoadmulee. namespace you are here 4 45 ready for pypi Let’s test this. Staying at the IDLE shell, import your module, create the list, and then try to invoke the function without a qualifying name. You’re expecting to see an error message: >>> import nester >>> cast = ['Palin', 'Cleese', 'Idle', 'Jones', 'Gilliam', 'Chapman'] >>> print_lol(cast) Traceback (most recent call last): File "", line 1, in print_lol(cast) NameError: name 'print_lol' is not defined ANsamexepEercrtoerd,, byeocuarusceodyoeuhdasidcna’tusqedualaify the name. When you qualify the name of the function with the namespace, things improve dramatically: >>> nester.print_lol(cast) Palin Cleese Idle Jones Gilliam Tthheislitstimaer, ethdiinsgpslawyeodrkonasscerxepeenc.ted…the data items in Chapman Geek Bits When you use a plain import statement, such as import nester, the Python interpreter is instructed to allow you to access nester’s functions using namespace qualification. However, it is possible to be more specific. If you use from nester import print_lol, the specified function (print_lol in this case) is added to the current namespace, effectively removing the requirement for you to use namespace qualification. But you need to be careful. If you already have a function called print_lol defined in your current namespace, the specific import statement overwrites your function with the imported one, which might not be the behavior you want. Your module is now ready for upload to PyPI. 46 Chapter 2 sharing your code Register with the PyPI website In order to upload your distribution to PyPI, you need to register with the PyPI website. This is a relatively straightforward process. Begin by surfing over to the PyPI website at http://pypi.python.org/ and requesting a PyPI ID: Provide the you’d like to Username use. Etwnticeerfyoorurcocnhfoisremnatpaiosnsword purposes. Dp(urononlv’etisdsiwnyogoruaryaPcaGtbuPoaulktlyeyhave one). tDhoant’tUtserryntaomeusies a“hlrfepaydtyhotna,k”ebn.ecause Provide a valid email address. “bcDIuliatocgntkr’oitnenge.f”oocrnhgteecthketbRooxecglbiiscetkfeortrhee If all of your registration details are in order, a confirmation message is sent to the email address submitted on the registration form. The email message contains a link you can click to confirm your PyPI registration: You are now registered with PyPI. Click your the PyPI croengfisitrmraattiioonn. link to complete you are here 4 47 register and upload Upload your code to PyPI You’re ready to rock! The code in your function has been placed in a module, used to create a distribution and installed into your local copy of Python. To upload your distribution to PyPI, complete these two steps: command-line registration with PyPI and command-line uploading. It might seem strange to have to register with PyPI again, seeing as you just did this with their website. However, the command-line uploading tool needs to be made aware of your PyPI Username and Password, and that’s what this registration does. Don’t worry: you have to do this only once. Instruct register setup to your details. Confirm that you want to use your just-created PyPI credentials. Use your PyPI settings and save them for future use. File Edit Window Help Register $ python3 setup.py register running register running check We need to know who you are, so please choose either: 1. use your existing login, 2. register as a new user, 3. have the server generate a new password for you (and email it to you), or 4. quit Your selection [default 1]: 1 Username: hfpython Password: Registering nester to http://pypi.python.org/pypi Server response (200): OK I can store your PyPI login so future submissions will be faster. (the login will be stored in /Users/barryp/.pypirc) Save your login (y/N)?y With your registration details entered and saved, you are now ready to upload your distribution to PyPI. Another command line does the trick: “aNnlerosettaeed:ry”If,tyayokoueu’nllt. gr§eyttaonueprloraodr aasmtohdautlencaamllee’sd IundpsitlsotrarudicbtuytosiueortnusptoofttoPwyaPrIe. Setup confirms that the upload is successful. Your distribution is now part of PyPI. File Edit Window Help Upload $ python3 setup.py sdist upload running sdist running check reading manifest file ‘MANIFEST’ creating nester-1.0.0 making hard links in nester-1.0.0... hard linking nester.py -> nester-1.0.0 hard linking setup.py -> nester-1.0.0 Creating tar archive removing ‘nester-1.0.0’ (and everything under it) running upload Submitting dist/nester-1.0.0.tar.gz to http://pypi.python.org/pypi Server response (200): OK $ 48 Chapter 2 Welcome to the PyPI community Congratulations! You are now a full-fledged, card-carrying member of the PyPI community. Your distribution has joined the over 10,000 other uploads on PyPI. Feel free to surf on over to the PyPI website to confirm the upload. Programmers from all over the globe are now able to download, unpack, and install your module into their local copy of Python, which is pretty cool when you think about it. Sit back, put your feet up, and wait for the plaudits to begin… sharing your code Yphououbw’lvisechoneodolwiysowturrhitactto?edne…and Q: Which is best: plain imports or specific imports? A: Neither, really. Most programmers mix and match based on their own personal preference and taste (although there are plenty of programmers willing to argue that their preferred way is the “one true way”). Note that the from module import function form pollutes your current namespace: names already defined in your current namespace are overwritten by the imported names. Q: And when I press F5 in IDLE’s edit window, it’s as if the module’s code is imported with an import statement, right? A: Yes, that is essentially what happens. The code in your edit window is compiled and executed by Python, and any names in the edit window are imported into the namespace being used by IDLE’s shell. This is handy, because it makes it easy to test functionality with IDLE. But bear in mind that outside of IDLE, you still need to import your module before you can use its functionality. Q: Is it really necessary for me to install my modules into my local copy of Python? Can’t I just put them in any old folder and import them from there? A: Yes, it is possible. Just bear in mind that Python looks for modules in a very specific list of places (recall the import sys; sys.path trick from earlier in this chapter). If you put your modules in a folder not listed in Python’s path list, chances are the interpreter won’t find them, resulting in ImportErrors. Using the distribution utilities to build and install your module into your local copy of Python avoids these types of errors. Q: I noticed the distribution utiliites created a file called nester.pyc. What’s up with that? A: That’s a very good question. When the interpreter executes your module code for the first time, it reads in the code and translates it into an internal bytecode format which is ultimately executed. (This idea is very similar to the way the Java JVM works: your Java code is turned into a class file as a result of your Java technologies compiling your code.) The Python interpreter is smart enough to skip the translation phase the next time your module is used, because it can determine when you’ve made changes to the original module code file. If your module code hasn’t changed, no translation occurs and the “compiled” code is executed. If your code has changed, the translation occurs (creating a new pyc file) as needed. The upshot of all this is that when Python sees a pyc file, it tries to use it because doing so makes everything go much faster. Q: Cool. So I can just provide my users with the pyc file? A: No, don’t do that, because the use of the pyc file (if found) is primarily a runtime optimization performed by the interpreter. Q: So, can I delete the pyc file if I don’t need it? A: Sure, if you really want to. Just be aware that you lose any potential runtime optimization. you are here 4 49 conflicting requests With success comes responsibility Lots of programmers from many different locations are using your module. And some of these programmers are looking for more features. We really love your code, but is there any chance this thing could print the data to screen and indent each nested list whenever one is found? Hang on a second. I kinda like the way it works right now. I vote NOT to change it. Likes done, what you’ve but could be happier. afAnunnnyocytcihotanhnigwseogsruktyso. the are way likely your to Requests for change are inevitable You need to keep your current users happy by maintaining the existing functionality, while at the same time providing enhanced functionality to those users that require it. This could be tricky. What are your options here? 50 Chapter 2 Life’s full of choices When it comes to deciding what to do here, there’s no shortage of suggestions. That’s soooo easy. Simply create another function called “print_lol2”, right? You could then import the function you want using the specific form of the import statement. It’s not that hard, really... Yeah, that might just work. You could edit your module’s code and define a new function called print_lol2, then code up the function to perform the nested printing When you want to use the original function, use this specific form of the import statement: from nester import print_lol. When you want to use the new, improved version of the function, use this import statement: from nester import print_lol2. Which would work, but… But that suggestion is twice the work...which might be OK sometimes...but the creation of a second, almost identical, function seems wasteful to me. sharing your code Right. A second function is wasteful. Not only are you introducing an almost identical function to your module, which might create a potential maintenance nightmare, but you’re also making things much more difficult for the users of your module, who must decide ahead of time which version of the function they need. Adding a second function makes your module’s application programming interface (API) more complex than it needs to be. There has to be a better strategy, doesn’t there? you are here 4 51 add an argument Control behavior with an extra argument If you add an extra argment to your function, you can handle indentation within your current code without too much trouble. Yikes! I should’ve thought about that myself... I probably need to go easy on the coffee. Of course, it’s clear to me now: adding another argument to your function gives you options. Take your function to the next level At the moment, your function has a single argument: the_list. If you add a second argument called level, you can use it to control indentation. A positive value for level indicates the number of tab-stops to include when displaying a line of data on screen. If level is 0, no indentation is used; if it’s 1, use a single tab-stop; if it’s 2, use two tab-stops; and so on. It’s clear you are looking at some sort of looping mechanism here, right? You already know how to iterate over a variably sized list, but how do you iterate a fixed number of times in Python? Does Python provide any functionality that can help? 52 Chapter 2 Before your write new code, think BIF When you come across a need that you think is generic, ask yourself if there’s a built-in function (BIF) that can help. After all, iterating a fixed number of times is something you’ll need to do all the time. And remember: Python 3 includes over 70 BIFs, so there’s a lot of functionality waiting to be discovered. sharing your code Use your pencil to draw a line matching each BIF to the correct description. The first one is done for you. Once you have all your lines drawn, circle the BIF you think you need to use in the next version of your function. BIF What the BIF does list() Creates a numbered list of paired-data, starting from 0. range() Returns the unique identification for a Python data object. enumerate() A factory function that creates a new, empty list. int() id() next() Returns the next item from a iterable data structure such as a list. Returns an iterator that generates numbers in a specified range on demand and as needed. Converts a string or another number to an integer (if possible). you are here 4 53 who did what SOLUTION You were to use your pencil to draw a line matching each BIF to the correct description. Once you had all your lines drawn, you were to circle the BIF you think you need to use in the next version of your function. BIF What the BIF does list() Creates a numbered list of paired-data, starting from 0. range() This BIF lionotkesresting. enumerate() Returns the unique identification for a Python data object. A factory function that creates a new, empty list. int() id() next() Returns the next item from a iterable data structure such as a list. Returns an iterator that generates numbers in a specified range on demand and as needed. Converts a string or another number to an integer (if possible). The range() BIF iterates a fixed number of times The range() BIF gives you the control you need to iterate a specific number of times and can be used to generate a list of numbers from zero upto-but-not-including some number. Here’s how to use it: “annunumdm”biseisrasstsghiegenneterdaartegeaedcthbidoyefn“trtaihfneigeer()” in turn. for num in range(4): print(num) 54 Chapter 2 bGuetn-ernaotte-innucmlubdeirnsg up-to4. Taphpeeanrumobnerscsr0ee,n1., 2, and 3 will sharing your code Q:Don’t I need to import the BIFs in order to use them in my program? A: No. For all intents and purposes, the BIFs are specifically imported into every Python program as well as IDLE. Q: So the BIFs must belong to the __main__ namespace, right? A: No. They are automatically imported into the __main__ namespace, but the BIFs have their very own namespace called (wait for it) __builtins__. Q: I get how range() works, but surely I could just as easily use a while loop to do the same thing? A: Yes, you can, but it’s not as elegant as using range(). Seriously, though, the while equivalent not only requires you to write more code, but it also makes it your responsibility to worry about loop state, whereas range() worries about this for you. As a general rule, Python programmers look for ways to reduce the amount of code they need to write and worry about, which leads to better code robustness, fewer errors, and a good night’s sleep. Q: So BIFs are actually good for me? A: BIFs exist to make your programming experience as straightforward as possible by providing a collection of functions that provide common solutions to common problems. Since they are included with Python, you are pretty much assured that they have been tested to destruction and do “exactly what it says on the tin.” You can depend on the BIFs. Using them gives your program a leg up and makes you look good. So, yes, the BIFs are good for you! Now that you know a bit about the range() BIF, amend your function to use range() to indent any nested lists a specific number of tab-stops. Hint: To display a TAB character on screen using the print() BIF yet avoid taking a new-line (which is print()’s default behavior), use this Python code: print("\t", end=''). """This is the "nester.py" module and it provides one function called print_lol() which prints lists that may or may not include nested lists.""" def print_lol(the_list, ): Include the name of the extra argument. """This function takes a positional argument called "the_list", which is any Python list (of - possibly - nested lists). Each data item in the provided list is (recursively) printed to the screen on it's own line.""" for each_item in the_list: if isinstance(each_item, list): print_lol(each_item) else: print(each_item) Don’t forget the comment. to edit Anudmdbceordoefhteraeb-tsototpask.e the required you are here 4 55 using range Now that you know a bit about the range() BIF, you were to amend your function to use range() to indent any nested lists a specific number of tab-stops. Hint: To display a TAB character on screen using the print() BIF yet avoid taking a new-line (which is print()’s default behavior), use this Python code: print("\t", end=''). """This is the "nester.py" module and it provides one function called print_lol() which prints lists that may or may not include nested lists.""" def print_lol(the_list, level ): """This function takes a positional argument called "the_list", which is any Python list (of - possibly - nested lists). Each data item in the provided list is (recursively) printed to the screen on it's own line.""" A second argument called “level" is used to insert tab-stops when a nested list is encountered.""" for each_item in the_list: if isinstance(each_item, list): print_lol(each_item) else: for tab_stop in range(level): Use how tmhaenyvatluaeb-ofsto“plesvealr”etuosecdo.ntrol print("\t", end='') print(each_item) oDfisipnldayenataTtAioBn.character for each level It’s time to test the new version of your function. Load your module file into IDLE, press F5 to import the function into IDLE’s namespace, and then invoke the function on your movies list with a second argument: >>> print_lol(movies, 0) aInvsoeckoenydouarrgfuumnecnttio.n, being sure to provide The Holy Grail 1975 Terry Jones & Terry Gilliam The data in “movies” starts to appear on screen… 91 Traceback (most recent call last): …then all hell breaks loose! Something is not right here. File "", line 1, in print_lol(movies,0) File "/Users/barryp/HeadFirstPython/chapter2/nester/nester.py", line 14, in print_lol print_lol(each_item) TypeError: print_lol() takes exactly 2 positional arguments (1 given) Here’s your clue as to what’s gone wrong. 56 Chapter 2 Your code has a TypeError, which caused it to crash. Python tries its best to run your code Unlike compiled languages (such as C, Java, C#, and others), Python doesn’t completely check the validity of your code until it runs. This allows Python to do some rather cool things that just aren’t possible in those other languages, such as dynamically defining functions at runtime. This, of course, can be very flexible and powerful. The cost to you, however, is that you need to be very careful when writing your code, because something that typically would be caught and flagged as an “error” by a traditional, statically typed, compiled language often goes unnoticed in Python. ...OK, C++ syntax fine...continuing to parse...whoops! You’re trying to use a function before it’s declared?!? That’s NOT allowed around here... I’m outta here. sharing your code Please wait. Compiling your C++ code… Ah ha! The old “calling a function before you’ve defined it yet” trick, eh? I’ll just make a note in case you define it later at runtime. You are planning to do that, right? Please don’t disappoint me, or I’ll give you an error... Running your Python code right now… Take another look at the error on the opposite page. Why do you think Python is giving you this particular error? What do you think is wrong? you are here 4 57 line check Trace your code When you’re trying to work out what went wrong with a program that looks like it should be OK, a useful technique is to trace what happens as each line of code executes. Here’s the code that you are currently working with. At only three lines long (remember: the creation of the list is one line of code), it doesn’t look like it should cause any trouble: import nester Thes two lines look OK. movies = [ "The Holy Grail", 1975, "Terry Jones & Terry Gilliam", 91,["Graham Chapman", ["Michael Palin", "John Cleese", "Terry Gilliam", "Eric Idle", "Terry Jones"]]] nester.print_lol(movies, 0) Ysooutahraet’isnvOoKki,ntgoto.he function with two arguments, With the data assigned to the function’s arguments, the function’s code starts to execute on each data item contained within the passed-in list: Teontsiarvee csopmacmee,ntthise not shown. def print_lol(the_list, level): “aTtshhseieg_n“lemisdto”vt,ioeas”“nldelivstetlh”ei.s assigned to value 0 is """This function ... """ Process each item in the list… …then decide what to do next based on dwahteathietremorisnaotlistth.e for each_item in the_list: if isinstance(each_item, list): print_lol(each_item) else: latlIoifhsosteket,cfhrroueiengncdhdcu,ttar!tits?oiahvnae…itltyheadimnnovgeiosskonaen’t for tab_stop in range(level): print("\t", end='') print(each_item) 58 Chapter 2 Work out what’s wrong There’s your problem: the recursive invocation of your function is using the old function signature that required only one argument. The new version of your function requires two arguments. The fix is easy: provide the correct number of arguments when calling the new version of your function. So, this snippet of code from your function: if isinstance(each_item, list): print_lol(each_item) needs to be rewritten to specify the correct number of arguments: if isinstance(each_item, list): print_lol(each_item, level) sharing your code Not so fast. Surely the nested list needs to be printed after a specific number of tab-stops? At the moment, your code sets “level” to 0 but never changes the value, so “level” never has any impact on your displayed output... Right. Your use of “level” needs one final tweak. The whole point of having level as an argument is to allow you to control the nested output. Each time you process a nested list, you need to increase the value of level by 1. Your code snippet needs to look like this: if isinstance(each_item, list): print_lol(each_item, level+1) It’s time to perform that update. Stiimmpelyyoiuncrreecmuernsitvetlyheinvvaolkuee yoofurlevfeulncbtyio1n.each you are here 4 59 fresh pypi Update PyPI with your new code Go ahead and edit your nester.py module (in the nester folder) to invoke your function properly. Now that you have a new version of your module, it’s a good idea to update the distribution that you uploaded to PyPI. With your code amended, there’s also a small change needed to your distribution’s setup.py program. You’ve changed your API, so adjust the value associated with version in setup.py. Let’s move from version 1.0.0 to 1.1.0: from distutils.core import setup setup( name version = 'nester', = '1.1.0', vtteaChorsihsssioianocindnsiga.iecitnaedttdeheeewdtviotaahlPunye“ePvweIrtsihoant” py_modules = ['nester'], author = 'hfpython', author_email = 'hfpython@headfirstlabs.com', url = 'http://www.headfirstlabs.com', description = 'A simple printer of nested lists', ) Just as you did when you created and uploaded your distribution, invoke the setup.py program within your distribution folder to perform the upload: Don’t you just love those “200 OK” messages? File Edit Window Help UploadAgain $ python3 setup.py sdist upload running sdist running check reading manifest file 'MANIFEST' creating nester-1.1.0 making hard links in nester-1.1.0... hard linking nester.py -> nester-1.1.0 hard linking setup.py -> nester-1.1.0 Creating tar archive removing 'nester-1.1.0' (and everything under it) running upload Submitting dist/nester-1.1.0.tar.gz to http://pypi.python.org/pypi Server response (200): OK $ Your new distribution is now available on PyPI. 60 Chapter 2 sharing your code Cool.There’s a new version of “nester” on PyPI. Take a look. Mark: Take a look at this, guys…the nester module has been updated on PyPI. Bob: Version 1.1.0… Laura: I wonder what’s changed? Mark: It still works with nested lists of lists, but now you can see the nested structure on screen, which I think is pretty cool. Laura: And useful. I’ve been waiting for that feature. Bob: Eh…OK…but how do I upgrade my existing local copy? Mark: Just follow the same steps as when you downloaded and installed nester from PyPI the first time. Bob: So I download the package file, unpack it, and ask setup.py to install it into my Python for me? Mark: Yes. It couldn’t be any easier. Laura: And what about my existing version of nester; what happens to that “old” version? Bob: Yeah…do I have two nester modules now? Bob Mark Laura Mark: No. When you use setup.py to install the latest version it becomes the current version and effectively replaces the previous module, which was the 1.0.0 release. Bob: And PyPI knows to give you the latest version of the module, too, right? Mark: Yes, when you surf the PyPI website and search for nester, you are always provided with the latest version of the module. Laura: Well, I use this module all the time and I’ve been waiting for this feature. I think I’ll update right away. Mark: I’ve already upgraded mine, and it works a treat. Bob: Yeah, I use it a lot, too, so I guess I’ll keep my system up to date and install the latest version. It’s probably not a good idea to rely on out-of-date software, right? Mark: I’d say. And, there’s nothing quite like progress. Laura: Catch you later, guys, I’ve got work to do. Bob: Me, too. I’m off to PyPI to grab the latest nester and install it into my local copy of Python. I’ll give it a quick test to confirm all is OK. Mark: Later, dudes… you are here 4 61 unhappy user You’ve changed your API Your new version of nester is indeed better, but not for all your users. Ah, phooey! I can’t believe it... I installed the latest version of “nester” from PyPI, and now all of my code that uses your function is not working. What did you do?!? Get with the program, Bob. You have TypeError’s everywhere... In your rush to release the lates and greatest version of your module, you forgot about some of your existing users. Recall that not all of your users want the new nested printing feature. However, by adding the second argument to print_lol(), you’ve changed your function’s signature, which means your module has a different API. Anyone using the old API is going to have problems. The ideal solution would be to provide both APIs, one which switches on the new feature and another that doesn’t. Maybe the feature could be optional? But how would that work? 62 Chapter 2 sharing your code Use optional arguments To turn a required argument to a function into an optional argument, provide the argument with a default value. When no argument value is provided, the default value is used. When an argument value is provided, it is used instead of the default. The key point is, of course, that the default value for the argument effectively makes the argument optional. To provide a default value for any function argument, specify the default value after the argument name: aBroetRhEaQrgUuImReEnDts. def print_lol(the_list, level): def print_lol(the_list, level=0): TdtOhuePrfeTnaaeuIddOltdN“ivtAlaeiLlvouenela”ohrfiagnsutamoeannt. With the default value for the argument defined, you can now invoke the function in a number of different ways: Ipnrvoovkiedethbeotfhunacrtgiuomn eanntds. nester.print_lol(movies, 0) sspbIetnrocavotorovhntkiddeianergatgarhvungeamulamfuelenutentnefcstrot,n.ribaotutntihvweeith nester.print_lol(movies) nester.print_lol(movies, 2) ttoInhhnveeeoskdaeerecgfotuanhmudelet.nftvuanalcuntediofnuosrwe ith Your function now supports different signatures, but the functonality remains as it was. you are here 4 63 idle session Amend your code to give the level argument a default value of 0 and then load your code into the IDLE editor. Press F5 to load the code into the shell and then follow along to confirm that the latest version of your function works as expected. Start be defining a short list of lists and use the function to display the the list on screen: >>> names = ['John', 'Eric', ['Cleese', 'Idle'], 'Michael', ['Palin']] >>> print_lol(names, 0) John Eric Cleese Idle Michael Tlaisshteesxsipntedacentndetader,ddw. bitehhanveisotredworks Palin Now try to do the same thing without specifiying the second argument. Let’s rely on the default value kicking in: >>> print_lol(names) John Eric Cleese Idle Michael Palin Without default isspuesceifdyainngdtwhoerskesc, otnodo.argument, the Now specify a value for the second argument and note the change in the function’s behavior: >>> print_lol(names, 2) John Eric Cleese Idle Michael Palin taShrpagetucmifleeynvetal.nanadltetrhneatinidveenvtailnuge sftoarrttshefrseocmond One final example provides what looks like a silly value for the second argument. Look what happens: >>> print_lol(names, -9) John Eric Cleese Idle Michael Uclioksuienngtthafeonorerg“iaglteinvivaeell”oviuastluupenulteikffeflreyocmttiovvebelyrescisoowmnite1c.a0he.p0so,sOirtFiigFvhetti?nhteeignedre.nTtihnigs, as the looks exactly Palin 64 Chapter 2 sharing your code Your module supports both APIs Well done! It looks like your module is working well, as both APIs, the original 1.0.0 API and the newer 1.1.0 API, can now be used. Let’s take a moment to create and upload a new distibution for PyPI. As before, let’s amend the version setting in the setup.py program: name version py_modules = 'nester', = '1.2.0', = ['nester'], “tvOehnrecseivoanal”gueaiinna,s“ssboeectiusauptr.epedyt”w.oitchhange And with the code changes applied, upload this new version of your distribution to PyPI: This all looks fine and dandy. File Edit Window Help UploadThree $ python3 setup.py sdist upload running sdist running check reading manifest file 'MANIFEST' creating nester-1.2.0 making hard links in nester-1.2.0... hard linking nester.py -> nester-1.2.0 hard linking setup.py -> nester-1.2.0 Creating tar archive removing 'nester-1.2.0' (and everything under it) running upload Submitting dist/nester-1.2.0.tar.gz to http://pypi.python.org/pypi Server response (200): OK $ Success! The messages from setup.py confirm that the your latest version of nester is up on PyPI. Let’s hope this one satisfies all of your users. Consider your code carefully. How might some of your users still have a problem with this version of your code? you are here 4 65 faulty default Your API is still not right Although the API lets your users invoke the function in its original form, the nesting is switched on by default. This behavior is not required by everyone and some people aren’t at all happy. Another version of “nester” has been released...but its default behavior might not be what you want. I can’t believe it! My programs were back to running fine, but now everything is indented. Has this thing changed again?!? Funny...it works fine for me. Of course, if you have some functionality that really ought to be optional (that is, not the default), you should adjust your code to make it so. But how? One solution is to add a third argument which is set to True when the indenting is required and False otherwise. If you ensure that this argument is False by default, the original functonality becomes the default behavior and users of your code have to request the new indenting feature explicitly. Let’s look at adding this final revision. 66 Chapter 2 sharing your code 1 Amend your module one last time to add a third argument to your function. Call your argument indent and set it initially to the value False—that is, do not switch on indentation by default. In the body of your function, use the value of indent to control your indentation code. Note: to save a bit of space, the comments from the module are not shown here. Of course, you need to make the necessary adjustments to your comments to keep them in sync with your code. Put the here. extra argument def print_lol(the_list, , level=0): for each_item in the_list: if isinstance(each_item, list): What go in needs here? to print_lol(each_item, , level+1) else: Atindoddceonanttliinrngoeloowcfchuecrnosd. e for tab_stop in range(level): print("\t", end='') print(each_item) 2 With your new code additions in place, provide the edit you would recommend making to the setup.py program prior to uploading this latest version of your module to PyPI: 3 Provide the command you would use to upload your new distribution to PyPI: you are here 4 67 adding an argument 1 You were to amend your module one last time to add a third argument to your function. You were to call your argument indent and set it initially to the value False—that is, do not switch on indentation by default. In the body of your function, you were to use the value of indent to control your indentation code. Did you default include value? the def print_lol(the_list, indent=False , for each_item in the_list: if isinstance(each_item, list): level=0): Ysioonuvbroecsaistguniroaent.utroeuhpadsatcehatnhgiesd, print_lol(each_item, indent , level+1) Astthsaeitmetpmrleiecnk“ti.fd”oes else: if indent : Don’t forget the colon at the end of the “if” line. for tab_stop in range(level): print("\t", end='') print(each_item) A is tswheisetcoadlet:erpnrainttiv(e"\tto" t*hisle“vfeol,re”nldo=op''). 2 With your new code additions in place, you were to provide the edit you would recommend making to the setup.py program prior to uploading this latest version of your module to PyPI: Edit “setup.py” so that it reads: version = ‘1.3.0’, It’s the vaalnueewavsseorcsiioatneodfwyiotuhr“mveordsuiolen,”sion byeousrur“esettoupc.hpayn”gfeile. 3 You were to provide the command you would use to upload your new distribution to PyPI: python3 setup.py sdisk upload Remember: instead of if you are “python3”. on Windows use “C:\Python31\python.exe” 68 Chapter 2 sharing your code A final test of the functonality should convince you that your module is now working exactly the way you and your users want it to. Let’s start with the original, default behavior: >>> names = ['John', 'Eric', ['Cleese', 'Idle'], 'Michael', ['Palin']] >>> print_lol(names) John Eric Cleese Idle TBisohrbee)s.otroirgeindal(,tdheaftaushltoufldunpctleiaosneality Michael Palin Next, turn on indentation by providing True as the second argument: >>> names = ['John', 'Eric', ['Cleese', 'Idle'], 'Michael', ['Palin']] >>> print_lol(names, True) John Eric Cleese Idle Michael Biotuy’tspppruootsvsi(idbklieenegptioangssweLcitoacnuhdraoanhragipunpmdyee)nn.tte, d Palin And, finally, control where indentation begins by providing a third argument value: >>> names = ['John', 'Eric', ['Cleese', 'Idle'], 'Michael', ['Palin']] >>> print_lol(names, True, 4) John Eric Cleese Idle Indenting tab-stop from is also a specific possible. Michael Palin Do this! Go ahead and edit your setup.py file; then upload your distribution to PyPI. you are here 4 69 one module for all Your module’s reputation is restored Congratulations! Word of your new and improved module is spreading fast. Great work! I love that I can switch indentation on and off. Lots of PyPI hits already. I told you this was good. My programs are back to working the way I want them to, so I’m a happy guy. Thanks! Thteoh’siashsiasmpaiplsey..cBl§ouste as Bob gets trust us, Your Python skills are starting to build You’ve created a useful module, made it shareable, and uploaded it to the PyPI website. Programmers all over the world are downloading and using your code in their projects. Keep up the good work. 70 Chapter 2 CHAPTER 2 Your Python Toolbox You’ve got Chapter 2 under your belt and you’ve added some key Python goodies to your toolbox. Python Lingo • a Umsueltaip“let-rilpinlee-cqoumomteedntstinrinygo”urtocoidncel.ude • “PyPI” is the is well worth a Python visit. Package Index and •mAem“onraymweshpearcee”naismeas place in exist. Python’s • Python’s main namespace is known as __main__. IDLE Notes • Press F5 to edit window. “run” the code in the IDLE ewnIn•cxDehaopeWemLddlnieehcEstieut’isonsnaliytnrnyu.ogeasoemutIsDpehtpesehLrpceeaiIEfsDcsi.iemcL.WFapETlo5iltyrhhsttihiisomnesiltp“slyo,aloorattuatehrdcemeo”dcenmonavidtnoeemtnd,oiuoeyldneocu’usele’s sharing your code A module is a text file that contains Python code. The distribution utilities let you turn your module into a shareable package. The setup.py program provides metadata about your module and is used to build, install, and upload your packaged distribution. Import your module into other programs using the import statement. Each module in Python provides its own namespace, and the namespace name is used to qualify the module’s functions when invoking them using the module. function() form. Specifically import a function from a module into the current namespace using the from module import function form of the import statement. Use # to comment-out a line of code or to add a short, one-line comment to your program. The built-in functions (BIFs) have their own namespace called __builtins__, which is automatically included in every Python program. The range() BIF can be used with for to iterate a fixed number of times. Including end=’’ as a argument to the print() BIF switches off its automatic inclusion of a new-line on output. Arguments to your functions are optional if you provide them with a default value. you are here 4 71 3 files and exceptions Dealing with errors I always thought he was exceptional...especially when it comes to processing my files. It’s simply not enough to process your list data in your code. You need to be able to get your data into your programs with ease, too. It’s no surprise then that Python makes reading data from files easy. Which is great, until you consider what can go wrong when interacting with data external to your programs…and there are lots of things waiting to trip you up! When bad stuff happens, you need a strategy for getting out of trouble, and one such strategy is to deal with any exceptional situations using Python’s exception handling mechanism showcased in this chapter. this is a new chapter 73 getting data in Data is external to your program Most of your programs conform to the input-process-output model: data comes in, gets manipulated, and then is stored, displayed, printed, or transferred. Dloattsaocfompleascefsr. om DBMS I’m ready for your data...just give it to me, baby! So far, you’ve learned how to process data as well as display it on screen. But what’s involved in getting data into your programs? Specifically, what’s involved in reading data from a file? How does Python read data from a file? 74 Chapter 3 It’s all lines of text The basic input mechanism in Python is line based: when read into your program from a text file, data arrives one line at a time. Python’s open() BIF lives to interact with files. When combined with a for statement, reading files is straightforward. Your data file called i“nskaettcehx.ttxt”. open() files and exceptions Your data as individual lines. When you use the open() BIF to access your data in a file, an iterator is created to feed the lines of data from your file to your code one line at a time. But let’s not get ahead of ourselves. For now, consider the standard openprocess-close code in Python: Do this! Open… …Process… …Close. the_file = open('sketch.txt') # Do something with the data # in "the_file". the_file.close() Create a folder called HeadFirstPython and a subfolder called chapter3. With the folders ready, download sketch.txt from the Head First Python support website and save it to the chapter3 folder. Let’s use IDLE to get a feel for Python’s file-input mechanisms. you are here 4 75 idle session Start a new IDLE sesson and import the os module to change the current working directory to the folder that contains your just-downloaded data file: >>> import os Import “os” from the Standard Library. >>> os.getcwd() What’s the current working directory? '/Users/barryp/Documents' >>> os.chdir('../HeadFirstPython/chapter3') Change to the folder that contains your data file. >>> os.getcwd() Confirm you are now in the right place. '/Users/barryp/HeadFirstPython/chapter3' Now, open your data file and read the first two lines from the file, displaying them on screen: >>> data = open('sketch.txt') Open a named file and assign the file to a file object called “data”. >>> print(data.readline(), end='') Man: Is this the right room for an argument? >>> print(data.readline(), end='') Other Man: I've told you once. UlBinsIeFe fttrhooemd“irstephalaedylfiniitlee(,o)”nthmsecenrteuhesoned. tthoe grab a “print()” Let’s “rewind” the file back to the start, then use a for statement to process every line in the file: >>> data.seek(0) 0 Use And the yes, “seek()” you can method to use “tell()” return to the with Python’s start files, of the too. file. >>> for each_line in data: print(each_line, end='') Titheirsatcoiodneusshinoguldthleoofkilef’asmdilaiatra: aits’sinapustt.andard Man: Is this the right room for an argument? Other Man: I've told you once. Man: No you haven't! Other Man: Yes I have. Man: When? Other Man: Just now. Man: No you didn't! Edfoivsreprlsaypyaleicdneeoronefassctorhneese,ndita(tailsathisough abridged here). ... Man: (exasperated) Oh, this is futile!! (pause) Other Man: No it isn't! Man: Yes it is! >>> data.close() Since you are now done with the file, be sure to close it. 76 Chapter 3 files and exceptions Take a closer look at the data Look closely at the data. It appears to conform to a specific format: The cast member’s role A colon, followed by a space character The cast line spoken member by the Man: Is this the right room for an argument? Other Man: I’ve told you once. Man: No you haven’t! Other Man: Yes I have. Man: When? Other Man: Just now. Man: No you didn’t! With this format in mind, you can process each line to extract parts of the line as required. The split() method can help here: Man: Is this the right room for an argument? Ianswtsvshrooeikcnneigeavttaehenrdeda“wbs“irpt:el”hiatik(st)ht”fehomeu“neesdtath.crohind_gline” each_line.split(":") This what tteollssp“lsitpliotn(.)” Man Is this the right room for an argument? The split() method returns a list of strings, which are assigned to a list of target identifiers. This is known as multiple assignment: A list of target identifiers on the left… …are assigned the strings returned by “split()”. (role, line_spoken) = each_line.split(":") aUsssiinggnetdhetheexastmrpinleg d“aMtaanf”,rowmhearbeoavse…, “role” is …t“hliener_igshptokreono”m: isfoarssiagnneadrgtuhmeensttr?i”ng “Is this Well? Is it? § you are here 4 77 idle session Let’s confirm that you can still process your file while splitting each line. Type the following code into IDLE’s shell: >>> data = open('sketch.txt') Open the data file. >>> for each_line in data: (role, line_spoken) = each_line.split(':') print(role, end='') print(' said: ', end='') Pearcohcelsisnethaenddadtisap,laeyxitnrgacetacinhg each part part from on screen. print(line_spoken, end='') Man said: Is this the right room for an argument? Other Man said: I've told you once. Man said: No you haven't! Other Man said: Yes I have. Man said: When? Other Man said: Just now. This all looks OK. Man said: No you didn't! Other Man said: Yes I did! Man said: You didn't! Other Man said: I'm telling you, I did! Man said: You did not! Other Man said: Oh I'm sorry, is this a five minute argument, or the full half hour? Man said: Ah! (taking out his wallet and paying) Just the five minutes. Other Man said: Just the five minutes. Thank you. Other Man said: Anyway, I did. Man said: You most certainly did not! Traceback (most recent call last): File "", line 2, in (role, line_spoken) = each_line.split(':') Wwrhoonogpsh!eTreh.ere’s something seriously ValueError: too many values to unpack It’s a ValueError, so that must mean there’s something wrong with the data in your file, right? 78 Chapter 3 Know your data Your code worked fine for a while, then crashed with a runtime error. The problem occurred right after the line of data that had the Man saying, “You most certainly did not!” Let’s look at the data file and see what comes after this successfully processed line: files and exceptions M a n : You didn't! Other Man: I'm telling you, I did! Man: You did not! Other Man: Oh I'm sorry, is this a five minute argument, or the full half hour? Man: Ah! (taking out his wallet and paying) Just the five minutes. Other Man: Just the five minutes. Thank you. Other Man: Anyway, I did. The error occurs AFTER this line Man: You most certainly did not! of data. Other Man: Now let's get one thing quite clear: I most definitely told you! Man: Oh no you didn't! Other Man: Oh yes I did! Notice anything? Notice anything about the next line of data? The next line of data has two colons, not one. This is enough extra data to upset the split() method due to the fact that, as your code currently stands, split()expects to break the line into two parts, assigning each to role and line_spoken, respectively. When an extra colon appears in the data, the split() method breaks the line into three parts. Your code hasn’t told split() what to do with the third part, so the Python interpreter raises a ValueError, complains that you have “too many values,” and terminates. A runtime error has occurred. What approach might you take to solve this dataprocessing problem? Do this! To help diagnose this problem, let’s put your code into its own file called sketch.py. You can copy and paste your code from the IDLE shell into a new IDLE edit window. you are here 4 79 ask for help Know your methods and ask for help It might be useful to see if the split() method includes any functionality that might help here. You can ask the IDLE shell to tell you more about the split() method by using the help() BIF. >>> help(each_line.split) Help on built-in function split: aLnoookpstiloikneal“asprlgitum()e”ntt.akes split(...) S.split([sep[, maxsplit]]) -> list of strings Return a list of the words in S, using sep as the delimiter string. If maxsplit is given, at most maxsplit splits are done. If sep is not specified or is None, any whitespace string is a separator and empty strings are removed from the result. The optional argument to split() controls how many breaks occur within your line of data. By default, the data is broken into as many parts as is possible. But you need only two parts: the name of the character and the line he spoke. If you set this optional argument to 1, your line of data is only ever broken into two pieces, effectively negating the effect of any extra colon on any line. Let’s try this and see what happens. Geek Bits IDLE gives you searchable access to the entire Python documentation set via its Help ➝ Python Docs menu option (which will open the docs in your web browser). If all you need to see is the documentation associated with a single method or function, use the help() BIF within IDLE’s shell. 80 Chapter 3 split(beans) split(beans, 1) files and exceptions Here’s the code in the IDLE edit window. Note the extra argument to the split() method. Tscpohlniettsre.oxltsrhaoawrg“uspmleitn(t)” With the edit applied and saved, press F5 (or select Run Module from IDLE’s Run menu) to try out this version of your code: >>> ================================ RESTART ================================ >>> Man said: Is this the right room for an argument? Other Man said: I've told you once. Man said: No you haven't! Other Man said: Yes I have. Man said: When? Other Man said: Just now. ... Taimbhrpeioddrgtiseapdnlattyoestdaulfolofuwttptouhtfeiits on this page. Other Man said: Anyway, I did. Man said: You most certainly did not! Other Man said: Now let's get one thing quite clear: I most definitely told you! Man said: Oh no you didn't! Other Man said: Oh yes I did! Man said: Oh no you didn't! Other Man said: Oh yes I did! Man said: Oh look, this isn't an argument! lCinoeolw. iYtohutmwaodceoiltonps…ast the …AbNutOTyoHuErRjoVyaliusesEhrorrotr!l!ived. There’s Traceback (most recent call last): File "/Users/barryp/HeadFirstPython/chapter4/sketch.py", line 5, in (role, line_spoken) = each_line.split(':', 1) ValueError: need more than 1 value to unpack That’s enough to ruin your day. What could be wrong now? you are here 4 81 missing colon Know your data (better) Your code has raised another ValueError, but this time, instead of complaining that there are “too many values,” the Python interpreter is complaining that it doesn’t have enough data to work with: “need more than 1 value to unpack.” Hopefully, another quick look at the data will clear up the mystery of the missing data. Other Man: Now let's get one thing quite clear: I most definitely told you! Man: Oh no you didn't! Other Man: Oh yes I did! Man: Oh no you didn't! Other Man: Oh yes I did! Man: Oh look, this isn't an argument! (pause) Other Man: Yes it is! Man: No it isn’t! (pause) tWhheaetx’spetchtise?d!?foSrommaet…ofwhtihceh data can’t dboeesgno’otdc. onform to Man: It's just contradiction! Other Man: No it isn't! The case of the missing colon Some of the lines of data contain no colon, which causes a problem when the split() method goes looking for it. The lack of a colon prevents split() from doing its job, causes the runtime error, which then results in the complaint that the interpreter needs “more than 1 value.” It looks like you still have problems with the data in your file. What a shame it’s not in a standard format. 82 Chapter 3 Two very different approaches When you have to deal with a bunch of exceptional situations, the best approach is to add extra logic. If there’s more stuff to worry about, you need more code. Jill files and exceptions Or you could decide to let the errors occur, then simply handle each error if and when it happens. That would be exceptional. Joe Jill’s suggested approach certainly works: add the extra logic required to work out whether it’s worth invoking split() on the line of data. All you need to do is work out how to check the line of data. Joe’s approach works, too: let the error occur, spot that it has happened, and then recover from the runtime error…somehow. Which approach works best here? you are here 4 83 find the substring Add extra logic Let’s try each approach, then decide which works best here. In addition to the split() method, every Python string has the find() method, too. You can ask find() to try and locate a substring in another string, and if it can’t be found, the find() method returns the value -1. If the method locates the substring, find() returns the index position of the substring in the string. Assign a string to the each_line variable that does not contain a colon, and then use the find() method to try and locate a colon: >>> each_line = "I tell you, there's no such thing as a flying circus." >>> each_line.find(':') -1 The string does NOT contain a colon, so “find()” returns -1 for NOT FOUND. Press Alt-P twice to recall the line of code that assigns the string to the variable, but this time edit the string to include a colon, then use the find() method to try to locate the colon: >>> each_line = "I tell you: there's no such thing as a flying circus." >>> each_line.find(':') 10 The string DOES contain a colon, so “find()” returns a positive index value. And you thought this approach wouldn’t work? Based on this IDLE session, I think this could do the trick. 84 Chapter 3 files and exceptions Adjust your code to use the extra logic technique demonstrated on the previous page to deal with lines that don’t contain a colon character. data = open('sketch.txt') What condition needs to go here? for each_line in data: if (role, line_spoken) = each_line.split(':', 1) print(role, end='') print(' said: ', end='') print(line_spoken, end='') data.close() Can you think of any potential problems with this technique? Grab your pencil and write down any issues you might have with this approach in the space provided below: you are here 4 85 substring found You were to adjust your code to use the extra logic technique to deal with lines that don’t contain a colon character: data = open('sketch.txt') for each_line in data: if not each_line.find(':') == -1: dhIteoaetsdawkaeorsrokua.ndfetwhisseccoonnddsittioon,gebtutyoitur “Nnntooehttgee”atckteoehsnyedtwiuhtoseireodnvo,a.flwuehtihcoehf (role, line_spoken) = each_line.split(':', 1) print(role, end='') print(' said: ', end='') print(line_spoken, end='') data.close() Iatltor’onsegtOdhaKifesfsiteef.hreeynyotua.rrJeisusssutimessiloar You were to think of any potential problems with this technique, grabbing your pencil to write down any issues you might have with this approach. There might be a problem with this code if the format of the data file changes, which will require changes to the condition. The condition used by the if statement is somewhat hard to read and understand. This code is a little “fragile”…it will break if another exceptional situation arises. 86 Chapter 3 files and exceptions Test Drive Amend your code within IDLE’s edit window, and press F5 to see if it works. tNhoisetrirmoer.s Your program works…although it is fragile. If the format of the file changes, your code will need to change, too, and more code generally means more complexity. Adding extra logic to handle exceptional situations works, but it might cost you in the long run. Maybe it’s time for a different approach? One that doesn’t require extra logic, eh? you are here 4 87 exceptional catch Handle exceptions Have you noticed that when something goes wrong with your code, the Python interpreter displays a traceback followed by an error message? The traceback is Python’s way of telling you that something unexpected has occurred during runtime. In the Python world, runtime errors are called exceptions. Whoooah! I don’t know what to do with this error, so I’m gonna raise an exception...this really is someone else’s problem. Oliokoeht, hyeurcek’!s It a looks bug. >>> if not each_ Traceback (most r File “>> ================================ RESTART ================================ >>> Man said: Is this the right room for an argument? Other Man said: I’ve told you once. Man said: No you haven’t! Other Man said: Yes I have. Man said: When? ... Tnohirsucnotdimeeweorrrkos,rsa,neditthheerr. e are OOther Man said: Nonsense! Man said: (exasperated) Oh, this is futile!! Other Man said: No it isn’t! Man said: Yes it is! So...both approaches work. But which is better? 94 Chapter 3 files and exceptions Tonight’s talk: Approaching runtime errors with extra code and exception handlers Extra Code: By making sure runtime errors never happen, I keep my code safe from tracebacks. Complexity never hurt anyone. I just don’t get it. You’re more than happy for your code to explode in your face…then you decide it’s probably a good idea to put out the fire?!? But the bad things still happen to you. They never happen with me, because I don’t let them. Well…that depends. If you’re smart enough—and, believe me, I am—you can think up all the possible runtime problems and code around them. Hard work never hurt anyone. Of course all my code is needed! How else can you code around all the runtime errors that are going to happen? Um, uh…most of them, I guess. Look: just cut it out. OK? Exception Handler: At the cost of added complexity…. I’ll be sure to remind you of that the next time you’re debugging a complex piece of code at 4 o’clock in the morning. Yes. I concentrate on getting my work done first and foremost. If bad things happen, I’m ready for them. Until something else happens that you weren’t expecting. Then you’re toast. Sounds like a whole heap of extra work to me. You did hear me earlier about debugging at 4 AM, right? Sometimes I think you actually enjoy writing code that you don’t need… Yeah…how many? You don’t know, do you? You’ve no idea what will happen when an unknown or unexpected runtime error occurs, do you? you are here 4 95 more errors What about other errors? It is true that both approaches work, but let’s consider what happens when other errors surface. OK, guys. How does you code react when the data file is deleted? Ummm...I’m not totally sure, to be honest... I guess that’ll produce another exception, but what happens in my code is anyone’s guess. Handling missing files Frank’s posed an interesting question and, sure enough, the problem caused by the removal of the data file makes life more complex for Jill and Joe. When the data file is missing, both versions of the program crash with an IOError. 96 Chapter 3 Do this! Rename the data file, then run both versions of your program again to confirm that they do indeed raise an IOError and generate a traceback. Add more error-checking code… If you’re a fan of the “let’s not let errors happen” school of thinking, your first reaction will be to add extra code to check to see if the data file exists before you try to open it, right? Let’s implement this strategy. Python’ s os module has some facilities that can help determine whether a data file exists, so we need to import it from the Standard Library, then add the required check to the code: files and exceptions Cthheecfkilwe heextihstesr. import os if os.path.exists('sketch.txt'): data = open('sketch.txt') All of this code remains unchanged. for each_line in data: if not each_line.find(':') == -1: (role, line_spoken) = each_line.split(':', 1) print(role, end='') print(' said: ', end='') print(line_spoken, end='') Inform the user of the bad news. data.close() else: print('The data file is missing!') A quick test of the code confirms that this new problem is dealt with properly. With this new version of your code in IDLE’s edit window, press F5 to confirm all is OK. >>> ================================ RESTART ================================ >>> The data file is missing! Exactly what was expected. Cool. >>> you are here 4 97 take it to another level …Or add another level of exception handling If you are a fan of the “handle exceptions as they occur” school of thinking, you’ll simply wrap your code within another try statement. Add “try” another statement. try: data = open('sketch.txt') Aots hweirthprtohgeram, all of this code remains unchanged. for each_line in data: try: (role, line_spoken) = each_line.split(':', 1) print(role, end='') print(' said: ', end='') print(line_spoken, end='') except: pass Give the the bad user news. data.close() except: print('The data file is missing!') Another quick test is required, this time with the version of your program that uses exception handling. Press F5 to give it a spin. >>> ================================ RESTART ================================ >>> The data file is missing! >>> Aprsoegxrpaemctheadn,dtlehsistvheersmioinssionfg the file, too. 98 Chapter 3 So, which approach is best? Well…it depends on who you ask! Here are both versions of your code: files and exceptions TtohihsavnedrlseioFnilueseIs/Oexetrrraorlos.gic TsthaitsevmeernsitontousheasnadnloetFheilre “try” I/O errors. Let’s ask a simple question about these two versions of your program: What do each of these programs do? Grab your pencil. In box 1, write down what you think the program on the left does. In box 2, write down what you think the program on the right does. 1 2 you are here 4 99 keep it simple You were to grab your pencil, then in box 1, write down what you thought the program on the left does. In box 2, write down what Tmphrooerrveeid’ssepdaacoelontftotrhoeywopurrreitvedio,eussscorpyiapogtueio.ancttuhaallny need was you thought the program on the right does. 1 The code on the right starts by importing the “os” library, and then it uses “path.exists” to make sure the data file exists, before it attempts to open the data file. Each line from the file is then processed, but only after it has determined that the line conforms to the required format by checking first for a single “:” character in the line. If the “:” is found, the line is processed; otherwise, it’s ignored. When we’re all done, the data file is closed. And you get a friendly message at the end if the file is not found. Now…that’s more like it. 2 The code on the right opens a data file, processes each line in that file, extracts the data of interest and displays it on screen. The file is closed when done. If any exceptions occur, this code handles them. Complexity is rarely a good thing Do you see what’s happening here? As the list of errors that you have to worry about grows, the complexity of the “add extra code and logic” solution increases to the point where it starts to obscure the actual purpose of the program. This is not the case with the exceptions handling solution, in which it’s obvious what the main purpose of the program is. By using Python’s exception-handling mechanism, you get to concentrate on what your code needs to do, as opposed to worrying about what can go wrong and writing extra code to avoid runtime errors. Prudent use of the try statement leads to code that is easier to read, easier to write, and—perhaps most important—easier to fix when something goes wrong. Concentrate on what your code needs to do. 100 Chapter 3 You’re done…except for one small thing Your exception-handling code is good. In fact, your code might be too good in that it is too general. At the moment, no matter what error occurs at runtime, it is handled by your code because it’s ignored or a error message is displayed. But you really need to worry only about IOErrors and ValueErrors, because those are the types of exceptions that occurred earlier when your were developing your program. Although it is great to be able to handle all runtime errors, it’s probably unwise to be too generic…you will want to know if something other than an IOError or ValueError occurs as a result of your code executing at runtime. If something else does happen, your code might be handling it in an inappropriate way. files and exceptions try: data = open('sketch.txt') for each_line in data: try: (role, line_spoken) = each_line.split(':', 1) print(role, end='') print(' said: ', end='') print(line_spoken, end='') except: pass data.close() except: Trishunibsteicimnogedeterrarineoddr.tohcicsucrosdweitrhuinnstwhheencoAdNe Ythat print('The data file is missing!') As your code is currently written, it is too generic. Any runtime error that occurs is handled by one of the except suites. This is unlikely to be what you want, because this code has the potential to silently ignore runtime errors. You need to somehow use except in a less generic way. you are here 4 101 specify exceptions Be specific with your exceptions If your exception-handling code is designed to deal with a specific type of error, be sure to specify the error type on the except line. In doing so, you’ll take your exception handling code from generic to specific. try: data = open('sketch.txt') for each_line in data: try: (role, line_spoken) = each_line.split(':', 1) print(role, end='') print(' said: ', end='') print(line_spoken, end='') except ValueError: pass data.close() except IOError: eSrpreocrifyyouthaeretyhpaendolfingr.untime print('The data file is missing!') Of course, if an different type of runtime error occurs, it is no longer handled by your code, but at least now you’ll get to hear about it. When you are specific about the runtime errors your code handles, your programs no longer silently ignore some runtime errors. Using “try/except” lets you concentrate on what your code needs to do... ...and it lets you avoid adding unnecessary code and logic to your programs. That works for me! 102 Chapter 3 Your Python Toolbox You’ve got Chapter 3 under your belt and you’ve added some key Python techiques to your toolbox. Python Lingo •ot•AfrdtnaAeahcs“aec“retbrtuxainrphcctateakcipsim.oetonbeiocaoecncfru”krr”otorceirhscd,eua.prrrsduoneadttsuiacmaiilenergdeesraurlotr IDLE Notes ydIob•DouycrALuccEmfhca’eoesvnosostHsriaenPittlgpyeiotPmnwhyeoetsnnbehu’tso.bnrdTshooDhowceuouslcemdPsrey.onftptrheaoontmniion3n files and exceptions Use the open() BIF to open a disk file, creating an iterator that reads data from the file one line at a time. The readline() method reads a single line from an opened file. The seek() method can be used to “rewind” a file to the beginning. The close() method closes a previously opened file. The split() method can break a string into a list of parts. An unchangeable, constant list in Python is called a tuple. Once list data is assigned to a tuple, it cannot be changed. Tuples are immutable. A ValueError occurs when your data does not conform to an expected format. An IOError occurs when your data cannot be accessed properly (e.g., perhaps your data file has been moved or renamed). The help() BIF provides access to Python’s documentation within the IDLE shell. The find() method locates a specific substring within another string. The not keyword negates a condition. The try/except statement provides an exception-handling mechanism, allowing you to protect lines of code that might result in a runtime error. The pass statement is Python’s empty or null statement; it does nothing. you are here 4 103 CHAPTER 3 4 persistence Saving data to files I’m in a bit of a pickle...my data is not as persistent as it could be. It is truly great to be able to process your file-based data. But what happens to your data when you’re done? Of course, it’s best to save your data to a disk file, which allows you to use it again at some later date and time. Taking your memory-based data and storing it to disk is what persistence is all about. Python supports all the usual tools for writing to files and also provides some cool facilities for efficiently storing Python data. So…flip the page and let’s get started learning them. this is a new chapter 105 save your work Programs produce data It’s a rare program that reads data from a disk file, processes the data, and then throws away the processed data. Typically, programs save the data they process, display their output on screen, or transfer data over a network. Your data is ready. Where should I put it for you? DBMS fYaiboleouuthtoavwuehseilco.htstoyfpecohfoicdeissk Hey, that’s nice output...exactly what I want. Before you learn what’s involved in writing data to disk, let’s process the data from the previous chapter to work out who said what to whom. When that’s done, you’ll have something worth saving. 106 Chapter 4 Code Magnets Add the code magnets at the bottom of this page to your existing code to satisfy the following requirements: 1. Create an empty list called man. 2. Create an empty list called other. 3. Add a line of code to remove unwanted whitespace from the line_spoken variable. 4. Provide the conditions and code to add line_spoken to the correct list based on the value of role. 5. Print each of the lists (man and other) to the screen. try: data = open('sketch.txt') for each_line in data: try: (role, line_spoken) = each_line.split(':', 1) persistence except ValueError: pass data.close() except IOError: print('The datafile is missing!') Here are your magnets. if role == 'Man': line_spoken = line_spoken.strip() elif role == 'Other Man': man = [] other = [] other.append(line_spoken) print(other) man.append(line_spoken) print(man) you are here 4 107 process and print Code Magnets Solution Your were to add the code magnets to your existing code to satisfy the following requirements: 1. Create an empty list called man. 2. Create an empty list called other. 3. Add a line of code to remove unwanted whitespace from the line_spoken variable. 4. Provide the conditions and code to add line_spoken to the correct list based on the value of role. 5. Print each of the lists (man and other) to the screen. man = [] other = [] Assign an “man” and empty list “other”. to try: data = open('sketch.txt') for each_line in data: try: Assign the bstarcikppoendtostirtisneglf. (role, line_spoken) = each_line.split(':', 1) line_spoken = line_spoken.strip() if role == 'Man': man.append(line_spoken) Trwehhmeito“evssetpsraiucpne(w)f”arnmotmeetdahosdtring. “elif” means “else if.” elif role == 'Other Man': other.append(line_spoken) bUapsdedatoenownehoofsatidhewlhisatts. except ValueError: pass data.close() except IOError: print('The datafile is missing!') print(man) print(other) Cproonccelussdeedbdyatdaispolnaysicnrgetenh.e 108 Chapter 4 persistence Test Drive Load your code into IDLE’s edit window and take it for a spin by pressing F5. Be sure to save your program into the same folder that contains sketch.txt. The edit code in window IDLE’s Aotnnhdescthrweeroeen’l:sisttwhsh.eatcoanptpeenatrss of It worked, as expected. Surely Python’s open() BIF can open files for writing as well as reading, eh? Yes, it can. When you need to save data to a file, the open() BIF is all you need. you are here 4 109 open and close Open your file in write mode When you use the open() BIF to work with a disk file, you can specify an access mode to use. By default, open() uses mode r for reading, so you don’t need to specify it. To open a file for writing, use mode w: The access to use model The data file object out = open("data.out", "w") The name of the file to write to By default, the print() BIF uses standard output (usually the screen) when displaying data. To write data to a file instead, use the file argument to specify the data file object to use: print("Norwegian Blues stun easily.", file=out) What gets written to the file When you’re done, be sure to close the file to ensure all of your data is written to disk. This is known as flushing and is very important: The file onbamjeectoftothweritdeattao out.close() This when iws rVitEinRgYtiomfpoilrets.ant Geek Bits When you use access mode w, Python opens your named file for writing. If the file already exists, it is cleared of its contents, or clobbered. To append to a file, use access mode a, and to open a file for writing and reading (without clobbering), use w+. If you try to open a file for writing that does not already exist, it is first created for you, and then opened for writing. 110 Chapter 4 persistence man = [] other = [] At the bottom of your program, two calls to the print() BIF display your processed data on screen. Let’s amend this code to save the data to two disk files instead. Call your disk files man_data.txt (for what the man said) and other_data.txt (for what the other man said). Be sure to both open and close your data files, as well as protect your code against an IOError using try/except. try: data = open('sketch.txt') for each_line in data: try: (role, line_spoken) = each_line.split(':', 1) line_spoken = line_spoken.strip() if role == 'Man': man.append(line_spoken) elif role == 'Other Man': other.append(line_spoken) except ValueError: pass data.close() except IOError: print('The datafile is missing!') Go on, try. Open data your files two here. Be sure to close your files. print(man, print(other, ) ) Swpreitceiftyotwhehefnilyeosutoinvoke “print()”. Handle any exceptions here. you are here 4 111 save to disk man = [] other = [] At the bottom of your program, two calls to the print() BIF display your processed data on screen. You were to amend this code to save the data to two disk files instead. You were to call your disk files man_data.txt (for what the man said) and other_data.txt (for what the other man said). You were to make sure to both open and close your data files, as well as protect your code against an IOError using try/except. try: data = open('sketch.txt') for each_line in data: try: (role, line_spoken) = each_line.split(':', 1) line_spoken = line_spoken.strip() if role == 'Man': man.append(line_spoken) All of this unchanged. code is elif role == 'Other Man': other.append(line_spoken) except ValueError: pass data.close() except IOError: print('The datafile is missing!') try: man_file = open(‘man_data.txt’, ‘w’) other_file = open(‘other_data.txt’, ‘w’) Did your yfoiulerseimneWmbReITr Etomoopdeen? Open your two files, and assign each to file objects. print(man, file=man_file ) print(other, file=other_file ) nUasmeetdhelis“tpsrtinot(n)a”mBedIFdtisok save the files. man_file.close() other_file.close() except IOError: print('File error.’) Don’t forget to close BOTH files. Handle an one occur. I/O exception, should 112 Chapter 4 persistence Test Drive Perform the edits to your code to replace your two print() calls with your new file I/O code. Then, run your program to confirm that the data files are created: ytBoheuefrroefroealrdyeeorun,rojpudrsatotgayroafumirlercsuoindnse,. yWohuerncyoodue,rtunhis iwasnhaal“tlemyloopoutkysse”leiI,kDeLE shell. ArCufRntEse,ArtTwyEoouDrnepiwnroyfgoirluearsmfaorelder. That code worked, too. You’ve created two data files, each holding the data from each of your lists. Go ahead and open these files in your favorite editor to confirm that they contain the data you expect. Consider the following carefully: what happens to your data files if the second call to print() in your code causes an IOError? you are here 4 113 close files Files are left open after an exception! When all you ever do is read data from files, getting an IOError is annoying, but rarely dangerous, because your data is still in your file, even though you might be having trouble getting at it. It’s a different story when writing data to files: if you need to handle an IOError before a file is closed, your written data might become corrupted and there’s no way of telling until after it has happened. try: Ok. man_file = open('man_data.txt', 'w') OK other_file = open('other_data.txt', 'w') OK print(man, file=man_file) OK print(other, file=other_file) Not OK!! Crash! man_file.close() other_file.close() except IOError: OK print('File error.') These two lines of code DON’T get to run. OK Your exception-handling code is doing its job, but you now have a situation where your data could potentially be corrupted, which can’t be good. What’s needed here is something that lets you run some code regardless of whether an IOError has occured. In the context of your code, you’ll want to make sure the files are closed no matter what. 114 Chapter 4 Extend try with finally When you have a situation where code must always run no matter what errors occur, add that code to your try statement’s finally suite: persistence No changes here, except that… …the calls to “close()” are moved to here. try: man_file = open('man_data.txt', 'w') other_file = open('other_data.txt', 'w') print(man, file=man_file) print(other, file=other_file) except IOError: print('File error.') finally: man_file.close() other_file.close() If no runtime errors occur, any code in the finally suite executes. Equally, if an IOError occurs, the except suite executes and then the finally suite runs. No matter what, the code in the finally suite always runs. By moving your file closing code into your finally suite, you are reducing the possibility of data corruption errors. This is a big improvement, because you’re now ensuring that files are closed properly (even when write errors occur). But what about those errors? How do you find out the specifics of the error? you are here 4 115 no dumb questiohns Q: I’m intrigued. When you stripped the line_spoken data of unwanted whitespace, you assigned the result back to the line_spoken variable. Surely invoking the strip() method on line_spoken changed the string it refers to? A: No, that’s not what happens. Strings in Python are immutable, which means that once a string is created, it cannot be changed. Q: But you did change the line_spoken string by removing any unwanted whitespace, right? A: Yes and no. What actually happens is that invoking the strip() method on the line_spoken string creates a new string with leading and trailing whitespace removed. The new string is then assigned to line_spoken, replacing the data that was referred to before. In effect, it is as if you changed line_ spoken, when you’ve actually completely replaced the data it refers to. Q: So what happens to the replaced data? A: Python’s built-in memory management technology reclaims the RAM it was using and makes it available to your program. That is, unless some other Python data object is also referring to the string. Q: What? I don’t get it. A: It is conceivable that another data object is referring to the string referred to by line_spoken. For example, let’s assume you have some code that contains two variables that refer to the same string, namely “Flying Circus.” You then decide that one of the variables needs to be in all UPPERCASE, so you invoke the upper() method on it. The Python interperter takes a copy of the string, converts it to uppercase, and returns it to you. You can then assign the uppercase data back to the variable that used to refer to the original data. Q: And the original data cannot change, because there’s another variable referring to it? A: Precisely. That’s why strings are immutable, because you never know what other variables are referring to any particular string. Q: But surely Python can work out how many variables are referring to any one particular string? A: It does, but only for the purposes of garbage collection. If you have a line of code like print('Flying Circus'), the string is not referred to by a variable (so any variable reference counting that’s going on isn’t going to count it) but is still a valid string object (which might be referred to by a variable) and it cannot have its data changed under any circumstances. Q: So Python variables don’t actually contain the data assigned to them? A: That’s correct. Python variables contain a reference to a data object.The data object contains the data and, because you can conceivably have a string object used in many different places throughout your code, it is safest to make all strings immutable so that no nasty side effects occur. Q: Isn’t it a huge pain not being able to adjust strings “in place”? A: No, not really. Once you get used to how strings work, it becomes less of an issue. In practice, you’ll find that this issue rarely trips you up. Q: Are any other Python data types immutable? A: Yes, a few. There’s the tuple, which is an immutable list. Also, all of the number types are immutable. Q: Other than learning which is which, how will I know when something is immutable? A: Don’t worry: you’ll know. If you try to change an immutable value, Python raises a TypeError exception. Q: Of course: an exception occurs. They’re everywhere in Python, aren’t they? A: Yes. Exceptions make the world go ’round. 116 Chapter 4 Knowing the type of error is not enough When a file I/O error occurs, your code displays a generic “File Error” message. This is too generic. How do you know what actually happened? Maybe the problem is that you can’t open the file? It could be that the file can be opened but not written to? persistence Yeah, or it could be a permission error, or maybe your disk is full? Who knows? It turns out that the Python interpreter knows…and it will give up the details if only you’d ask. When an error occurs at runtime, Python raises an exception of the specific type (such as IOError, ValueError, and so on). Additionally, Python creates an exception object that is passed as an argument to your except suite. Let’s use IDLE to see how this works. you are here 4 117 idle session Let’s see what happens when you try to open a file that doesn’t exist, such as a disk file called missing.txt. Enter the following code at IDLE’s shell: >>> try: data = open('missing.txt') print(data.readline(), end='') except IOError: print('File error') finally: data.close() File error There’s your error message, but… Traceback (most recent call last): File "", line 7, in data.close() NameError: name 'data' is not defined …rwaihsaetd’satndhisi?t!?kiAllendotyhoeurr exception code. was As the file doesn’t exist, the data file object wasn’t created, which subsequently makes it impossible to call the close() method on it, so you end up with a NameError. A quick fix is to add a small test to the finally suite to see if the data name exists before you try to call close(). The locals() BIF returns a collection of names defined in the current scope. Let’s exploit this BIF to only invoke close() when it is safe to do so: finally: The for “in” operator membership. tests if 'data' in locals(): data.close() Tendeheitids syisotujorusctchoatdnhegeea.btPitIrDeosLfsEAc’olstds-ehPetlhlt.aot File error No extra Just your eerxrcoerptmioensssatghei.s time. Here you’re searching the collection returned by the locals() BIF for the string data. If you find it, you can assume the file was opened successfully and safely call the close() method. If some other error occurs (perhaps something awful happens when your code calls the print() BIF), your exception-handling code catches the error, displays your “File error” message and, finally, closes any opened file. But you still are none the wiser as to what actually caused the error. 118 Chapter 4 persistence When an exception is raised and handled by your except suite, the Python interpreter passes an exception object into the suite. A small change makes this exception object available to your code as an identifier: except IOError as err: print('File error: ' + err) Give your a name… exception object …ytohuerneursreoritmaessspaagret. of But when you try to run your code with this change made, another exception is raised: Traceback (most recent call last): File "", line 5, in print('File error:' + err) TypeError: Can't convert 'IOError' object to str implicitly “WeTxhycopeoeppEtsir!ornYo;ertt”h. aisnottimheerit’s a This time your error message didn’t appear at all. It turns out exception objects and strings are not compatible types, so trying to concatenate one with the other leads to problems. You can convert (or cast) one to the other using the str() BIF: except IOError as err: print('File error: ' + str(err)) eUxsceeptthieon“sotbrj(e)c”tBtIoFbteohafvoercliekethaestring. Now, with this final change, your code is behaving exactly as expected: File error: [Errno 2] No such file or directory: 'missing.txt' Amwnhedsastaygoweuentntohwawtrgtoenetgll.as specific error you exactly Of course, all this extra logic is starting to obscure the real meaning of your code. Wouldn’t it be dreamy if there were a way to take advantage of these mechanisms without the code bloat? I guess it’s just a fantasy... you are here 4 119 try with Use with to work with files Because the use of the try/except/finally pattern is so common when it comes to working with files, Python includes a statement that abstracts away some of the details. The with statement, when used with files, can dramatically reduce the amount of code you have to write, because it negates the need to include a finally suite to handle the closing of a potentially opened data file. Take a look: try: data = open('its.txt', "w") print("It's...", file=data) except IOError as err: print('File error: ' + str(err)) finally: if 'data' in locals(): data.close() Texhcisepits /tfhienaullsyu”alp“attrtye/rn. Tntehhgeeatu“sfeesinotaflhlye“”wnseiuteihdt”ef. or try: with open('its.txt', "w") as data: print("It's...", file=data) except IOError as err: print('File error: ' + str(err)) When you use with, you no longer have to worry about closing any opened files, as the Python interpreter automatically takes care of this for you. The with code on the the right is identical in function to that on the left. At Head First Labs, we know which approach we prefer. Geek Bits The with statement takes advantage of a Python technology called the context management protocol. 120 Chapter 4 Write your “with” code here. persistence Grab your pencil and rewrite this try/except/finally code to use with instead. Here’s your code with the appropriate finally suite added: try: man_file = open('man_data.txt', 'w') other_file = open('other_data.txt', 'w') print(man, file=man_file) print(other, file=other_file) except IOError as err: print('File error: ' + str(err)) finally: if 'man_file' in locals(): man_file.close() if 'other_file' in locals(): other_file.close() you are here 4 121 no finally You were to grab your pencil and rewrite this try/except/finally code to use with instead. Here’s your code with the appropriate finally suite added: try: man_file = open('man_data.txt', 'w') other_file = open('other_data.txt', 'w') print(man, file=man_file) print(other, file=other_file) except IOError as err: print('File error: ' + str(err)) finally: if 'man_file' in locals(): man_file.close() if 'other_file' in locals(): other_file.close() try: with open(‘man_data.txt', ‘w') as man_file: “Ustfthsiainentagecllmoytd”ewneostuwsi“tiwtteiho.tohru”etwtrihtee print(man, file=man_file) with open(‘other_data.txt', ‘w') as other_file: print(other, file=other_file) except IOError as err: print(‘File error: ' + str(err)) “Owritcho”msbtianteemtheentt.wo “open()” calls into one Note the use of the comma. with open('man_data.txt', 'w') as man_file, open('other_data.txt’, 'w’) as other_file: print(man, file=man_file) print(other, file=other_file) 122 Chapter 4 persistence Test Drive Add your with code to your program, and let’s confirm that it continues to function as expected. Delete the two data files you created with the previous version of your program and then load your newest code into IDLE and give it a spin. Nstshohuecalelctrearstposhpfreeusalplirynrs.ottghoreaimnIDdriLcaaEnte If you check your folder, your two data files should’ve reappeared. Let’s take a closer look at the data file’s contents by opening them in your favorite text editor (or use IDLE). Here’s what man said. the Here’s other what the man said. You’ve saved the lists in two files containing what the Man said and what the Other man said. Your code is smart enough to handle any exceptions that Python or your operating system might throw at it. Well done. This is really coming along. you are here 4 123 unsuitable format Default formats are unsuitable for files Although your data is now stored in a file, it’s not really in a useful format. Let’s experiment in the IDLE shell to see what impact this can have. Use a with statement to open your data file and display a single line from it: >>> with open('man_data.txt') as mdf: print(mdf.readline()) bNeoctaeu:seno“wnietehd” tdooecslotsehaytoufrorfiyleo,u. ['Is this the right room for an argument?', "No you haven't!", 'When?', "No you didn't!", "You didn't!", 'You did not!', 'Ah! (taking out his wallet and paying) Just the five minutes.', 'You most certainly did not!', "Oh no you didn't!", "Oh no you didn't!", "Oh look, this isn't an argument!", "No it isn't!", "It's just contradiction!", 'It IS!', 'You just contradicted me!', 'You DID!', 'You did just then!', '(exasperated) Oh, this is futile!!', 'Yes it is!'] Yikes! It would appear your list is converted to a large string by print() when it is saved. Your experimental code reads a single line of data from the file and gets all of the data as one large chunk of text…so much for your code saving your list data. What are your options for dealing with this problem? Geek Bits By default, print() displays your data in a format that mimics how your list data is actually stored by the Python interpreter. The resulting output is not really meant to be processed further… its primary purpose is to show you, the Python programmer, what your list data “looks like” in memory. 124 Chapter 4 I guess I could write some custom parsing code to process the “internal format” used by “print()”. It shouldn’t take me all that long... persistence It might be worth looking at using something other than a plain “print()” to format the data prior to saving it to the data file? I’d certainly look into it. Parsing the data in the file is a possibility…although it’s complicated by all those square brackets, quotes, and commas. Writing the required code is doable, but it is a lot of code just to read back in your saved data. Of course, if the data is in a more easily parseable format, the task would likely be easier, so maybe the second option is worth considering, too? Can you think of a function you created from earlier in this book that might help here? you are here 4 125 nonstandard output Why not modify print lol()? Recall your print_lol() function from Chapter 2, which takes any list (or list of lists) and displays it on screen, one line at a time. And nested lists can be indented, if necessary. This functionality sounds perfect! Here’s your code from the nester.py module (last seen at the end of Chapter 2): Tdhatisacoondethceursrcernetelny. displays your Amending this code to print to a disk file instead of the screen (known as standard output) should be relatively straightforward. You can then save your data in a more usable format. 126 Chapter 4 Standard Output The default place where your code writes its data when the “print()” BIF is used. This is typically the screen. In Python, standard output is referred to as “sys.stdout” and is importable from the Standard Library’s “sys” module. persistence 1 Let’s add a fourth argument to your print_lol() function to identify a place to write your data to. Be sure to give your argument a default value of sys.stdout, so that it continues to write to the screen if no file object is specified when the function is invoked. Fill in the blanks with the details of your new argument. (Note: to save on space, the comments have been removed from this cod, but be sure to update your comments in your nester.py module after you’ve amended your code.) def print_lol(the_list, indent=False, level=0, ): for each_item in the_list: if isinstance(each_item, list): print_lol(each_item, indent, level+1, else: if indent: for tab_stop in range(level): print("\t", end='', print(each_item, ) ) ) 2 What needs to happen to the code in your with statement now that your amended print_lol() function is available to you? 3 List the name of the module(s) that you now need to import into your program in order to support your amendments to print_lol(). you are here 4 127 extend your function 1 You were to add a fourth argument to your print_lol() function to identify a place to write your data to, being sure to give your argument a default value of sys.stdout so that it continues to write to the screen if no file object is specified when the function is invoked. You were to fill in the blanks with the details of your new argument. (Note: to save on space, the comments have been removed from this code, but be sure to update those in your nester.py module after you’ve amended your code). Addefdautlhtevfaoluuer.th argument and give it a def print_lol(the_list, indent=False, level=0, fh=sys.stdout ): for each_item in the_list: if isinstance(each_item, list): fh print_lol(each_item, indent, level+1, else: Note: the ) signature changed. has if indent: for tab_stop in range(level): print("\t", end='', file=fh print(each_item, file=fh ) ) Actaodljluusssettott“hhpeernitnewtwo()” argument. 2 What needs to happen to the code in your with statement now that your amended print_lol() function is available to you? The code needs to be adjusted so that instead of using the “print()” BIF, the code needs to invoke “print_lol()” instead. 3 List the name of the module(s) that you now need to import into your program in order to support your amendments to print_lol(). The program needs to import the amended “nester” module. 128 Chapter 4 persistence Test Drive Before taking your code for a test drive, you need to do the following: 1. Make the necessary changes to nester and install the amended module into your Python environment (see Chapter 2 for a refresher on this). You might want to upload to PyPI, too. 2. Amend your program so that it imports nester and uses print_lol() instead of print() within your with statement. Note: your print_lol() invocation should look something like this: print_lol(man, fh=man_file). When you are ready, take your latest program for a test drive and let’s see what happens: As before, output on there’s screen. no Let’s check the contents of the files to see what they look like now. sWaihdatistnhoewman legible. This is looking good. By amending your nester module, you’ve provided a facility to save your list data in a legible format. It’s now way easier on the eye. But does this make it any easier to read the data back in? And here’s the other said. what man you are here 4 129 brittle code Hang on a second...haven’t you been here before? You’ve already written code to read in lines from a data file and put ‘em into lists...do you like going around in circles?!? That’s a good point. This problem is not unlike the problem from the beginning of the chapter, in that you’ve got lines of text in a disk file that you need to process, only now you have two files instead of one. You know how to write the code to process your new files, but writing custom code like this is specific to the format that you’ve created for this problem. This is brittle: if the data format changes, your custom code will have to change, too. Ask yourself: is it worth it? 130 Chapter 4 Custom Code Exposed This week’s interview: When is custom code appropriate? persistence Head First: Hello, CC, how are you today? Custom Code: Hi, I’m great! And when I’m not great, there’s always something I can do to fix things. Nothing’s too much trouble for me. Here: have a seat. Head First: Why, thanks. Custom Code: Let me get that for you. It’s my new custom SlideBack&Groove™, the 2011 model, with added cushions and lumbar support…and it automatically adjusts to your body shape, too. How does that feel? Head First: Actually [relaxes], that feels kinda groovy. Custom Code: See? Nothing’s too much trouble for me. I’m your “go-to guy.” Just ask; absolutely anything’s possible when it’s a custom job. Head First: Which brings me to why I’m here. I have a “delicate” question to ask you. Custom Code: Go ahead, shoot. I can take it. Head First: When is custom code appropriate? Custom Code: Isn’t it obvious? It’s always appropriate. Head First: Even when it leads to problems down the road? Custom Code: Problems?!? But I’ve already told you: nothing’s too much trouble for me. I live to customize. If it’s broken, I fix it. Head First: Even when a readymade solution might be a better fit? Custom Code: Readymade? You mean (I hate to say it): off the shelf? Head First: Yes. Especially when it comes to writing complex programs, right? Custom Code: What?!? That’s where I excel: creating beautifully crafted custom solutions for folks with complex computing problems. Head First: But if something’s been done before, why reinvent the wheel? Custom Code: But everything I do is custommade; that’s why people come to me… Head First: Yes, but if you take advantage of other coders’ work, you can build your own stuff in half the time with less code. You can’t beat that, can you? Custom Code: “Take advantage”…isn’t that like exploitation? Head First: More like collaboration, sharing, participation, and working together. Custom Code: [shocked] You want me to give my code…away? Head First: Well…more like share and share alike. I’ll scratch your back if you scratch mine. How does that sound? Custom Code: That sounds disgusting. Head First: Very droll [laughs]. All I’m saying is that it is not always a good idea to create everything from scratch with custom code when a good enough solution to the problem might already exist. Custom Code: I guess so…although it won’t be as perfect a fit as that chair. Head First: But I will be able to sit on it! Custom Code: [laughs] You should talk to my buddy Pickle…he’s forever going on about stuff like this. And to make matters worse, he lives in a library. Head First: I think I’ll give him a shout. Thanks! Custom Code: Just remember: you know where to find me if you need any custom work done. you are here 4 131 in a pickle Pickle your data Python ships with a standard library called pickle, which can save and load almost any Python data object, including lists. Once you pickle your data to a file, it is persistent and ready to be read into another program at some later date/time: YinouPrydtahtoan’sasmietmaoprypears The pickle engine ['Is this the right room for an argument?', "No you haven't!", 'When?', "No you didn't!", "You didn't!", 'You did not!', 'Ah! (taking out his wallet and paying) Just the five minutes.', 'You most certainly did not!', "Oh no you didn't!", "Oh no you didn't!", "Oh look, this isn't an argument!", "No it isn't!", "It's just contradiction!", 'It IS!', 'You just contradicted me!', 'You DID!', 'You did just then!', '(exasperated) Oh, this is futile!!', 'Yes it is!'] Feed data your Python to pickle. [‘Is this the right room for an argument?’, “No you haven’t!”, ‘When?’, “No you didn’t!”, “You didn’t!”, ‘You did not!’, ‘Ah! (taking out his wallet and paying) Just the five minutes.’, ‘You most certainly did not!’, “Oh no you didn’t!”, “Oh no you didn’t!”, “Oh look, this isn’t an argument!”, “No it isn’t!”, “It’s just contradiction!”, ‘It IS!’, ‘You just contradicted me!’, ‘You DID!’, ‘You did just then!’, ‘(exasperated) Oh, this is futile!!’, ‘Yes it is!’] Your pickled data Opiuctklecdomveesrstiohne of your data. You can, for example, store your pickled data on disk, put it in a database, or transfer it over a network to another computer. When you are ready, reversing this process unpickles your persistent pickled data and recreates your data in its original form within Python’s memory: The same pickle engine eYinxoauPcryttdlyhaotanas’sbismefreoemrcoerr.eyat, ed Your pickled data [‘Is this the right room for an argument?’, “No you haven’t!”, ‘When?’, “No you didn’t!”, “You didn’t!”, ‘You did not!’, ‘Ah! (taking out his wallet and paying) Just the five minutes.’, ‘You most certainly did not!’, “Oh no you didn’t!”, “Oh no you didn’t!”, “Oh look, this isn’t an argument!”, “No it isn’t!”, “It’s just contradiction!”, ‘It IS!’, ‘You just contradicted me!’, ‘You DID!’, ‘You did just then!’, ‘(exasperated) Oh, this is futile!!’, ‘Yes it is!’] Feed data tyooupricpkilcek.led OyPouyuttrhcoponimckveleserdtsihodenatoaf. ['Is this the right room for an argument?', "No you haven't!", 'When?', "No you didn't!", "You didn't!", 'You did not!', 'Ah! (taking out his wallet and paying) Just the five minutes.', 'You most certainly did not!', "Oh no you didn't!", "Oh no you didn't!", "Oh look, this isn't an argument!", "No it isn't!", "It's just contradiction!", 'It IS!', 'You just contradicted me!', 'You DID!', 'You did just then!', '(exasperated) Oh, this is futile!!', 'Yes it is!'] 132 Chapter 4 Save with dump and restore with load Using pickle is straightforward: import the required module, then use dump() to save your data and, some time later, load() to restore it. The only requirement when working with pickled files is that they have to be opened in binary access mode: Always remember to import the “pickle” module. Tuosesa“vdeumyopu(r)”.data, import pickle ... with open('mydata.pickle', 'wb') as mysavedata: pickle.dump([1, 2, 'three'], mysavedata) ... with open('mydata.pickle', 'rb') as myrestoredata: a_list = pickle.load(myrestoredata) Artseossitgaonnreyiddouedrnattifaier. print(a_list) fRielestuosriengyo“uloradd(a)t”a. from your Otrnecaetyioturlikdeataanyisobtahcekr in your program, data object. you can What if something goes wrong? If something goes wrong when pickling or unpickling your data, the pickle module raises an exception of type PickleError. persistence yiTPnoyhuBterIhN“dobnAa”tRtatYoeflolispleesn mode. w Here’s a snippet of your code as it currently stands. Grab your pencil and strike out the code you no longer need, and then replace it with code that uses the facilities of pickle instead. Add any additional code that you think you might need, too. try: with open('man_data.txt', 'w') as man_file, open('other_data.txt', 'w') as other_file: nester.print_lol(man, fh=man_file) nester.print_lol(other, fh=other_file) except IOError as err: print('File error: ' + str(err)) you are here 4 133 significance of the pickle print(‘Pickling error: ‘ + str(perr)) Here’s a snippet of your code as it currently stands. You were to grab import pickle try: ptIrmhoepgortraotmp .“opfickyloeu”rnear your pencil and strike out the code you no longer need, and then replace it with code that uses the facilities pickle instead. You were also to add any additional code that you think you might need. Cbeha“nwgreittehaeblaec,cbeisnsarmyo”d.e to ‘wb' 'wb' with open('man_data.txt', 'w') as man_file, open('other_data.txt', 'w') as other_file: nester.print_lol(man, fh=man_file) pickle.dump(man, man_file) nester.print_lol(other, fh=other_file) pickle.dump(other, other_file) except IOError as err: print('File error: ' + str(err)) except pickle.PickleError as perr: print('Pickling error: ' + str(perr)) Rweitphlaccealltshteot“wpoickcalell.ds utmop“(n)”e.ster.print_lol()” tDhoant’tcafnorogcecturt.o handle any exceptions Q: When you invoked print_lol() earlier, you provided only two arguments, even though the function signature requires you to provide four. How is this possible? A: When you invoke a Python function in your code, you have options, especially when the function provides default values for some arguments. If you use positional arguments, the position of the argument in your function invocation dictates what data is assigned to which argument. When the function has arguments that also provide default values, you do not need to always worry about positional arguments being assigned values. Q: OK, you’ve completely lost me. Can you explain? A: Consider print(), which has this signature: print(value, sep=' ', end='\n', file=sys.stdout). By default, this BIF displays to standard output (the screen), because it has an argument called file with a default value of sys.stdout. The file argument is the fourth positional argument. However, when you want to send data to something other than the screen, you do not need to (nor want to have to) include values for the second and third positional arguments. They have default values anyway, so you need to provide values for them only if the defaults are not what you want. If all you want to do is to send data to a file, you invoke the print() BIF like this: print("Dead Parrot Sketch", file='myfavmonty.txt') and the fourth positional argument uses the value you specify, while the other positional arguments use their defaults. In Python, not only do the BIFs work this way, but your custom functions support this mechamism, too. 134 Chapter 4 persistence Test Drive Let’s see what happens now that your code has been amended to use the standard pickle module instead of your custom nester module. Load your amended code into IDLE and press F5 to run it. Ogtenhtcaetnaosgovamiisenut,ahylioncugluehas happened. So, once again, let’s check the contents of the files to see what they look like now: Tpihckeleisdtdhaetam.an’s The is man’s ptichkeleodthdearta. It appears to have worked…but these files look like gobbledygook! What gives? Recall that Python, not you, is pickling your data. To do so efficiently, Python’s pickle module uses a custom binary format (known as its protocol). As you can see, viewing this format in your editor looks decidedly weird. Don’t worry: it is supposed to look like this. you are here 4 135 idle session pickle really shines when you load some previously pickled data into another program. And, of course, there’s nothing to stop you from using pickle with nester. After all, each module is designed to serve different purposes. Let’s demonstrate with a handful of lines of code within IDLE’s shell. Start by importing any required modules: >>> import pickle >>> import nester No surprises there, eh? Next up: create a new identifier to hold the data that you plan to unpickle.Create an empty list called new_man: >>> new_man = [] Yes, almost too exciting for words, isn’t it? With your list created. let’s load your pickled data into it. As you are working with external data files, it’s best if you enclose your code with try/except: >>> try: with open('man_data.txt', 'rb') as man_file: new_man = pickle.load(man_file) except IOError as err: print('File error: ' + str(err)) except pickle.PickleError as perr: print('Pickling error: ' + str(perr)) This code is not news to you either. However, at this point, your data has been unpickled and assigned to the new_man list. It’s time for nester to do its stuff: >>> nester.print_lol(new_man) Is this the right room for an argument? No you haven’t! When? No you didn’t! ... You did just then! (exasperated) Oh, this is futile!! Yes it is! thNheoertree,a.bllutthterudsattausi:sitsh’sowalnl And to finish off, let’s display the first line spoken as well as the last: >>> print(new_man[0]) Is this the right room for an argument? >>> print(new_man[-1]) Yes it is! 136 Chapter 4 See: after all that, it is the right room! § Generic file I/O with pickle is the way to go! Now, no matter what data you create and process in your Python programs, you have a simple, tested, triedand-true mechanism for saving and restoring your data. How cool is that? persistence Python takes care of your file I/O details, so you can concentrate on what your code actually does or needs to do. As you’ve seen, being able to work with, save, and restore data in lists is a breeze, thanks to Python. But what other data structures does Python support out of the box? Let’s dive into Chapter 5 to find out. you are here 4 137 CHAPTER 4 python toolbox Your Python Toolbox You’ve got Chapter 4 under your belt and you’ve added some key Python techiques to your toolbox. Python Lingo •ian“c•IhsvPmsaaa•“tyrmvPnlfoute“igrruineUshcetaogt,okdngmoanclap.eribani.ilptcnngdeekhng”arltaotis-anttyiasg,pts”hteooaeashbnv-n”veejcceede-etpcthtrasddhteosaastacotiptteorgarsaansopvgeoaectedoblre.yufjsspeeissectsotefnce 138 Chapter 4 The strip() method removes unwanted whitespace from strings. The file argument to the print() BIF controls where data is sent/saved. The finally suite is always executed no matter what exceptions occur within a try/except statement. An exception object is passed into the except suite and can be assigned to an identifier using the as keyword. The str() BIF can be used to access the stringed representation of any data object that supports the conversion. The locals() BIF returns a collection of variables within the current scope. The in operator tests for membership. The “+” operator concatenates two strings when used with strings but adds two numbers together when used with numbers. The with statement automatically arranges to close all opened files, even when exceptions occur. The with statement uses the as keyword, too. sys.stdout is what Python calls “standard output” and is available from the standard library’s sys module. The standard library’s pickle module lets you easily and efficiently save and restore Python data objects to disk. The pickle.dump() function saves data to disk. The pickle.load() function restores data from disk. CHAPTER 4 5 comprehending data Work that data! Life could be so much easier if only she’d let me help her extract, sort, and comprehend her data... Data comes in all shapes and sizes, formats and encodings. To work effectively with your data, you often have to manipulate and transform it into a common format to allow for efficient processing, sorting, and storage. In this chapter, you’ll explore Python goodies that help you work your data up into a sweat, allowing you to achieve data-munging greatness. So, flip the page, and let’s not keep the coach waiting… this is a new chapter 139 coaching crisis Coach Kelly needs your help I’m too busy on the track to waste time fiddling with my computer. Can you help me process my athlete data? The coach is an old friend, and you’d love to help. His crack squad of U10 athletes has been training hard. With each 600m run they do, Coach Kelly has recorded their time in a text file on his computer. There are four files in all, one each for James, Sarah, Julie, and Mikey. james.txt julie.txt 2-34,3:21,2.34,2.45,3.01,2:01,2:01,3:10,2-22 2.59,2.11,2:11,2:23,3-10,2-23,3:10,3.21,3-21 2:58,2.58,2:39,2-25,2-55,2:54,2.18,2:55,2:55 2:22,3.01,3:01,3.02,3:02,3.02,3:22,2.49,2:38 sarah.txt mikey.txt Initially, the coach needs a quick way to know the top three fastest times for each athlete. Can you help? 140 Chapter 5 Do this! Before proceeding with this chapter, take a few moments to download the four data files from the Head First Python support website. comprehending data Let’s begin by reading the data from each of the files into its own list. Write a short program to process each file, creating a list for each athlete’s data, and display the lists on screen. Hint: Try splitting the data on the commas, and don’t forget to strip any unwanted whitespace. Write your code here. you are here 4 141 let’s split Let’s begin by reading the data from each of the files into its own list. You were to write a short program to process each file, creating a list for each athlete’s data. You were then to display the lists on screen. Open the file. with open(‘james.txt’) as jaf: data = jaf.readline() Read the line of data. james = data.strip().split(‘,’) Convert the data to a list. with open(‘julie.txt’) as juf: Otinhpeetnudreanat,cahrefoaifldes tdfalhiainlelteeisa,ltoianffnferdrodoomafcmtrteata.hhteee data = juf.readline() julie = data.strip().split(‘,’) with open(‘mikey.txt’) as mif: data = mif.readline() mikey = data.strip().split(‘,’) with open(‘sarah.txt’) as saf: data = saf.readline() sarah = data.strip().split(‘,’) print(james) print(julie) print(mikey) print(sarah) Display the four lists on screen. Q: That data.strip().split(',') line looks a little weird. Can you explain what’s going on? A: That’s called method chaining. The first method, strip(), is applied to the line in data, which removes any unwanted whitespace from the string. Then, the results of the stripping are processed by the second method, split(','), creating a list. The resulting list is then applied to the target identifier in the previous code. In this way, the methods are chained together to produce the required result. It helps if you read method chains from left to right. 142 Chapter 5 Test Drive Load your code into IDLE and run it to confirm that it’s all OK for now: Here’s your program as displayed in IDLE. comprehending data Aprnoddhuecered’sbtyhreunonuitnpgut your code. So far, so good. Coach Kelly’s data is now represented by four lists in Python’s memory. Other than the use of method chaining, there’s nothing much new here, because you’ve pretty much mastered reading data from files and using it to populate lists. There’s nothing to show the coach yet, so no point in disturbing him until his data is arranged in ascending order, which requires you to sort it. Let’s look at your sorting options in Python. you are here 4 143 in-place or copied sorting Sort in one of two ways When it comes to sorting your data using Python, you have two options. In-place sorting takes your data, arranges it in the order you specify, and then replaces your original data with the sorted version. The original ordering is lost. With lists, the sort() method provides in-place sorting: Tunhoerodreirgeindald, ata [ 1, 3, 4, 2, 6, 5 ] TEnhgeinPeyttrhaonnsf“oInr-mpsl,atceheSnorretpinlagc”es. [ 1, 2, 3, 4, 5, 6 ] Trneohpwelabocerediegn)i.noarl ddeartead has (and Copied sorting takes your data, arranges it in the order you specify, and then returns a sorted copy of your original data. Your original data’s ordering is maintained and only the copy is sorted. In Python, the sorted() BIF supports copied sorting. TEnhgeinPeyttrhaonnsf“oCrompisedanSdorrettinugr”ns. Tunhoerodreirgeindald, ata [ 1, 3, 4, 2, 6, 5 ] [ 1, 3, 4, 2, 6, 5 ] TrehmeaoinrsigUinNalT, OunUoCrdHeErDed. data 144 Chapter 5 [ 1, 2, 3, 4, 5, 6 ] Torhdeedraetda(ahnads ncoowpiebde)e.n comprehending data Let’s see what happens to your data when each of Python’s sorting options is used. Start by creating an unordered list at the IDLE shell: >>> data = [6, 3, 1, 2, 4, 5] >>> data [6, 3, 1, 2, 4, 5] auCsnrsoiegrandteetroeadalidsvtaatroiaafbalned. Perform an in-place sort using the sort() method that is built in as standard to every Python list: >>> data.sort() >>> data [1, 2, 3, 4, 5, 6] Perform IN-PLACE sorting on the data. The data’s ordering has changed. Reset data to its original unordered state, and then perform a copied sort using the sorted() BIF: >>> data = [6, 3, 1, 2, 4, 5] >>> data [6, 3, 1, 2, 4, 5] >>> data2 = sorted(data) >>> data [6, 3, 1, 2, 4, 5] >>> data2 [1, 2, 3, 4, 5, 6] The data’s ordering has been reset. Perform COPIED sorting on the data. Same as it ever was. Tfrhoemcolopwieedstdattoahiisghoersdte. red Either sorting option works with the coach’s data, but let’s use a copied sort for now to arrange to sort the data on output. In the space below, provide four amended print() statements to replace those at the bottom of your program. you are here 4 145 all sorted “SysdoioamruttpadleydiBs(cpE)al”aFllyoOniRtEtohne screen. Either sorting option works with the coach’s data, but let’s use a copied sort for now to arrange to sort the data on output. You were to provide four amended print() statements to replace those at the bottom of your program. print(sorted(james)) print(sorted(julie)) print(sorted(mikey)) print(sorted(sarah)) Q: What happens to the unsorted data when I use sort()? Q: And there’s no way to get the original data back? A: A: For all intents and purposes, it disappears. Python takes a copy, No. If the ordering of the original data is important to you, use sorts it, and then replaces your original data with the sorted version. the sorted() BIF to transform your data into a sorted copy. 146 Chapter 5 Geek Bits You’ve already seen method chaining, and now it’s time to say “hello” to function chaining. Function chaining allows you to apply a series of functions to your data. Each function takes your data, performs some operation on it, and then passes the transformed data on to the next function. Unlike method chains, which read from left to right, function chains read from right to left (just to keep things interesting). comprehending data Test Drive Let’s see if this improves your output in any way. Make the necessary amendments to your code and run it. Hteoret’hsetchoeduep. dates wTsBoehriurettded.dloa…otkwahaiistchnToitHs,IaSlit!kea,ll LctohomaotkingiastBwtEehiFrisdO:.R2E-525.18is…now Hey, it looks like your data values are not uniform. Is the problem with all those periods, dashes, and colons? Yes. The minute and seconds separators are confusing Python’s sorting technology. When recording his athletes’ times in each of their files, Coach Kelly sometimes used a different character to separate minutes from seconds. It looks like you need to fix your data. you are here 4 147 time trials The trouble with time Well…there’s never enough of it, is there? Let’s look closely at the coach’s data to see what the problem is. Here’s Sarah raw data again: 2:58,2.58,2:39,2-25,2-55,2:54,2.18,2:55,2:55 sarah.txt Recall that data read from a file comes into your program as text, so Sarah’s data looks like this once you turn it into a list of “times”: Oh, look: what a lovely bunch of strings... ['2:58', '2.58', '2:39’, '2-25', '2-55', '2:54’, '2.18', '2:55', '2:55'] These thinks atrheeya’lrl esttriimngess,. even though the coach And when you sort Sarah’s data, it ends up in this order (which isn’t quite what you were expecting): ['2-25', '2-55', '2.18', '2.58', '2:39', '2:54', '2:55', '2:55', '2:58'] crWoigmhhoeto.paHsf!otTwerhca2atn-’s52n5.o1?8t W2ca.h5no’8topacsnodamge2aib:n5e.4t2w,:3ece9ann it? I don’t get what the problem is...they’re all times to me. Python sorts the strings, and when it comes to strings, a dash comes before a period, which itself comes before a colon. As all the strings start with 2, the next character in each string acts like a grouping mechanism, with the dashed times grouped and sorted, then the period times, and finally the colon times. Nonuniformity in the coach’s data is causing the sort to fail. 148 Chapter 5 Code Magnets Let’s create a function called sanitize(), which takes as input a string from each of the athlete’s lists. The function then processes the string to replace any dashes or colons found with a period and returns the sanitized string. Note: if the string already contains a period, there’s no need to sanitize it. Rearrange the code magnets at the bottom of the page to provide the required functionality. def sanitize(time_string): comprehending data return(mins + '.' + secs) tRheetucranlletrheofsatnhitisizfeudncttiimoen.string to Your magnets are waiting. elif ':' in time_string: time_string.split(splitter) else: if '-' in time_string: splitter = ':' splitter = '-' return(time_string) (mins, secs) = you are here 4 149 sanitzing function Code Magnets Solution You were to create a function called sanitize(), which takes as input a string from each of the athlete’s lists. The function then processes the string to replace any dashes or colons found with a period and returns the sanitized string. Note: if the string already contains a period, there’s no need to sanitize it. You were to rearrange the code magnets at the bottom of the previous page to provide the required functionality. def sanitize(time_string): Use the “in” operator to check string if the contains a dash or a colon. if '-' in time_string: splitter = '-' elif ':' in time_string: splitter = ':' else: return(time_string) NDOoTnonteheidngtoifbtehseanstitriizngedd. oes (mins, secs) = time_string.split(splitter) return(mins + '.' + secs) Smpinliutttesheansdtrsinegcotnodsexptarratcst. the Of course, on its own, the sanitize() function is not enough. You need to iterate over each of your lists of data and use your new function to convert each of the athlete’s times into the correct format. Let’s put your new function to work right away. 150 Chapter 5 comprehending data Let’s write the code to convert your existing data into a sanitized version of itself. Create four new lists to hold the sanitized data. Iterate over each athlete’s list data and append each sanitized string from each list to the appropriate new list. Conclude your program by printing a sorted copy of each new list to the screen. Trfehraoedmcsottdhheeetddhaaatttaa files remains (ucoanonncmdhtpahrhnieasgssespdebadegeetn)o. fit Acdoddeyhoeurre.new with open('james.txt') as jaf: data = jaf.readline() james = data.strip().split(',') with open('julie.txt') as juf: data = juf.readline() julie = data.strip().split(',') with open('mikey.txt') as mif: data = mif.readline() mikey = data.strip().split(',') with open('sarah.txt') as saf: data = saf.readline() sarah = data.strip().split(',') tWohtathehfaopuprens “print()” statements? print( print( print( print( ) ) ) ) you are here 4 151 sanitized for your protection Create four new, initially empty lists. Let’s write the code to convert your existing data into a sanitized version of itself. You were to create four new lists to hold the sanitized data. You were then to iterate over each athlete’s data and append each sanitized string from each list to an appropriate new list. You were to conclude your program by printing a sorted copy of each new list to the screen. with open('james.txt') as jaf: data = jaf.readline() james = data.strip().split(',') with open('julie.txt') as juf: data = juf.readline() julie = data.strip().split(',') with open('mikey.txt') as mif: data = mif.readline() mikey = data.strip().split(',') with open('sarah.txt') as saf: data = saf.readline() sarah = data.strip().split(',') clean_james = [] clean_julie = [] clean_mikey = [] clean_sarah = [] for each_t in james: clean_james.append(sanitize(each_t)) for each_t in julie: clean_julie.append(sanitize(each_t)) for each_t in mikey: clean_mikey.append(sanitize(each_t)) for each_t in sarah: clean_sarah.append(sanitize(each_t)) slTdsiotsirthsatpest,leaefdwmyo,heuttinrchotheo“sp.annrreioenwwt()” print( print( print( sorted(clean_james) sorted(clean_julie) sorted(clean_mikey) ) ) ) print( sorted(clean_sarah) ) ldaTtisnaahttdke.eatoetrhaioegcnithnhaaolepfplaiestpntpdhsr,eotsdpahraneitiatsataizeniteitneteimzwhseedmin, 152 Chapter 5 comprehending data Test Drive Combine your sanitize() function with your amended code from the previous page, and then press F5 in IDLE to confirm the sorting is now working as expected. Tcohnetsaainnsit‘i.z’ eads daasteapaornalytor. Ttihmeesoradreerninogwwcoormkps,arbaebclaeu.se all the This output looks much better. It’s taken a bit of work, but now the data from each of the four files is both sorted and uniformly formatted. By preprocessing your data before you sort it, you’ve helped ensure Python’s sorting technology performs correctly. Geek Bits By default, both the sort() method and the sorted() BIF order your data in ascending order. To order your data in descending order, pass the reverse=True argument to either sort() or sorted() and Python will take care of things for you. Four sorted lists you are here 4 153 duplicated code Hang on a sec! Something doesn’t feel quite right...look at all that duplicated code, as well as all those duplicated lists. This duplication is bad, right? Is this really the best you can do? That’s right. Duplicated code is a problem. As things stand, your code creates four lists to hold the data as read from the data files. Then your code creates another four lists to hold the sanitized data. And, of course, you’re iterating all over the place… There has to be a better way to write code like this. Transforming lists is such a common requirement that Python provides a tool to make the transformation as painless as possible. This tool goes by the rather unwieldly name of list comprehension. And list comprehensions are designed to reduce the amount of code you need to write when transforming one list into another. 154 Chapter 5 Comprehending lists Consider what you need to do when you transform one list into another. Four things have to happen. You need to: 1 Create a new list to hold the transformed data. 2 Iterate each data item in the original list. 3 With each iteration, perform the transformation. 4 Append the transformed data to the new list. comprehending data 1. Create. clean_mikey = [] 2. Iterate. for each_t in mikey: clean_mikey.append(sanitize(each_t)) 3. Transform. 4. Append. Here’s the same functionality as a list comprehension, which involves creating a new list by specifying the transformation that is to be applied to each of the data items within an existing list. rYideoegunultgaierftiiettreortapoticiukosnets)h(.jeusttarligkeetwith clean_mikey = [sanitize(each_t) for each_t in mikey] The new list is created… …by applying a transformation… …to each data item… What’s interesting is that the transformation has been reduced to a single line of code. Additionally, there’s no need to specify the use of the append() method as this action is implied within the list comprehension. Neat, eh? …within an existing list. you are here 4 155 idle session Let’s see some other list comprehension examples. Open up your IDLE shell and follow along with these one-liner transformations. Start by transforming a list of minutes into a list of seconds: >>> mins = [1, 2, 3] >>> secs = [m * 60 for m in mins] >>> secs [60, 120, 180] Simply minute multiply the values by 60. How about meters into feet? >>> meters = [1, 10, 3] >>> feet = [m * 3.281 for m in meters] >>> feet [3.281, 32.81, 9.843] Yfeese,ttihneraemaertee3r..281 Given a list of strings in mixed and lowercase, it’s a breeze to transform the strings to UPPERCASE: >>> lower = ["I", "don't", "like", "spam"] >>> upper = [s.upper() for s in lower] >>> upper ['I', "DON'T", 'LIKE', 'SPAM'] tEhveer“yupspterrin()g”cmometehsowdi.th Let’s use your sanitize() function to transform some list data into correctly formatted times: >>> dirty = ['2-22', '2:22', '2.22'] >>> clean = [sanitize(t) for t in dirty] >>> clean ['2.22', '2.22', '2.22'] Idti’rstnyevinetrobseoemnestohienagsyclteaont. u§rn something It’s also possible to assign the results of the list transformation back onto the original target identifier. This example transforms a list of strings into floating point numbers, and then replaces the original list data: >>> clean = [float(s) for s in clean] >>> clean [2.22, 2.22, 2.22] The “float()” BIF converts to floating point. And, of course, the transformation can be a function chain, if that’s what you need: >>> clean = [float(sanitize(t)) for t in ['2-22', '3:33', '4.44']] >>> clean [2.22, 3.33, 4.44] Citoemmbsinisinsguptproarntsfedor, mtaotoi!ons on the data 156 Chapter 5 comprehending data Now that you know about list comprehensions, let’s write four of them to process the coach’s four lists of timing values. Transform each of your lists into sorted, sanitized version of themselves. Grab your pencil and in the space provided, scribble the list comprehensions you plan to use. Q: So…let me get this straight: list comprehensions are good and list iterations are bad, right? A: No, that’s not the best way to look at it. If you have to perform a transformation on every item in a list, using a list comprehension is the way to go, especially when the transformation is easily specified on one line (or as a function chain). List iterations can do everything that list comprehensions can, they just take more code, but iterations do provide more flexibility should you need it. Geek Bits Python’s list comprehension is an example of the language’s support for functional programming concepts. There’s plenty of debate about the best way to develop program code: either procedurally, using functional programming techniques, or using object orientation. At Head First Labs, we try not to get involved in this debate, other than to rejoice in the fact that Python supports, in one way or another, all three of these programming practices. you are here 4 157 list comprehensions Tptnbehreyraewfntolslihrfissetmotrs“cmisostoamhrtttephioreeendnh,(e)oan”rnsddBioeInrtFeh.de Now that you know about list comprehensions, you were to write four of them to process the coach’s four lists of timing values. You were to transform each of your lists into sorted, sanitized version of themselves. You were to grab your pencil and in the space provided, scribble the list comprehensions you plan to use. sorted([sanitize(t) for t in james]) sorted([sanitize(t) for t in julie]) sorted([sanitize(t) for t in mikey]) sorted([sanitize(t) for t in sarah]) fRoirnstehaendotrheeprealitsts. Be careful about where you use the sorted() BIF when defining your list comprehensions. You may have been tempted to use the function chain sorted(sanitize(t)) within your list comprehension. Don’t be. Recall that the transformation works on one list item at a time, not the entire list. In this example, the sorted() BIF expects to sort a list, not an individual data item. The beauty of list comprehensions The use of list comprehensions with the coach’s athlete data has resulted in a lot less code for you to maintain. Additionally, as you get used to list comprehension syntax and usage, you’ll find that their use is natural and matches the way your brain thinks about your data and the transformations that you might want to apply. Let’s confirm that your new code is working as expected. 158 Chapter 5 comprehending data Test Drive Replace your list iteration code from earlier with your four new (beautiful) list comprehensions. Run your program to confirm that the results have not changed. YpslioarsumtordeiutnoceeeurwtaEptliuXisottAnsaCc.soTmyLopYurrehteehanersiloienrs As expected, the outout matches that from earlier. You’ve written a program that reads Coach Kelly’s data from his data files, stores his raw data in lists, sanitizes the data to a uniform format, and then sorts and displays the coach’s data on screen. And all in 25 lines of code. It’s probably safe to let the coach take a look at your output now. What will the coach think? you are here 4 159 list slicing Have you not been drinking enough water? I wanted the three fastest times for each athlete...but you’ve given me everything and it contains duplicates! In your haste to sanitize and sort your data, you forgot to worry about what you were actually supposed to be doing: producing the three fastest times for each athlete. And, of course, there’s no place for any duplicated times in your output. Accessing the first three data items from any list is easy. Either specify each list item individually using the standard notation or use a list slice: Ayocucenseseedacinhddivaidtuaailtlye.m james[0] james[1] james[2] james[0:3] But…what about removing duplicates from your list? 160 Chapter 5 Uitseema0lisutp-slticoe-btouta-cncoetss-ifnrcolumdilnisgt list item 3. Iterate to remove duplicates Processing a list to remove duplicates is one area where a list comprehension can’t help you, because duplicate removal is not a transformation; it’s more of a filter. And a duplicate removal filter needs to examine the list being created as it is being created, which is not possible with a list comprehension. To meet this new requirement, you’ll need to revert to regular list iteration code. comprehending data Assume that the fourth from last line of code from your current program is changed to this: james = sorted([sanitize(t) for t in james]) That is, instead of printing the sanitized and sorted data for James to the screen, this line of code replaces James’s unordered and nonuniform data with the sorted, sanitized copy. Your next task is to write some code to remove any duplicates from the james list produced by the preceding line of code. Start by creating a new list called unique_james, and then populate it with the unique data items found in james. Additionally, provide code to display only the top three fastest times for James. Hint: you might want to consider using the not in operator. you are here 4 161 top three Assume that the fourth from last line of code from your current program is changed to this: james = sorted([sanitize(t) for t in james]) That is, instead of printing the sanitized and sorted data for James to the screen, this line of code replaces James’s unordered and non-uniform data with the sorted, sanitized copy. Your next task was to write some code to remove any duplicates from the james list produced by the preceding line of code. You were to start by creating a new list called unique_james and then populate it with the unique data items found in james. Additionally, you were to provide code to only display the top three fastest times for James. Cehmorelpdattytehlietshtuentioque data items. unique_james = [] for each_t in james: Iterate existing over the data… …and if the data item ISN’T if each_t not in unique_james: already in the new list… unique_james.append(each_t) …tahpepennedwtlihset.unique data item to sdftScrihrlsiorpecmeeleaenytt.dhhtaeehteaflimsirtitsoteanmnds print(unique_james[0:3]) 162 Chapter 5 Do this! Repeat the code on this page for the rest of the coach’s lists: julie, mikey & sarah. Add all of your new code to your existing program. comprehending data Test Drive Take all of the recent amendments and apply them to your program. Run this latest code within IDLE when you are ready. Sort each and list. sanitize Remove the duplicates. Looking good! It worked! You are now displaying only the top three times for each athlete, and the duplicates have been successfully removed. The list iteration code is what you need in this instance. There’s a little bit of duplication in your code, but it’s not too bad, is it? you are here 4 163 duplicated code to remove duplication? “Not too bad”...you’re kidding, right?!? Surely there’s something that can be done with all that duplicated duplicate code? The irony is hard to avoid, isn’t it? The code that removes duplicates from your lists is itself duplicated. Sometimes such a situation is unavoidable, and sometimes creating a small function to factor out the duplicated code can help. But something still doesn’t feel quite right here… 164 Chapter 5 comprehending data Wouldn't it be dreamy if there were a way to quickly and easily remove duplicates from an existing list? But I know it's just a fantasy... you are here 4 165 factory functions Remove duplicates with sets In addition to lists, Python also comes with the set data structure, which behaves like the sets you learned all about in math class. The overriding characteristics of sets in Python are that the data items in a set are unordered and duplicates are not allowed. If you try to add a data item to a set that already contains the data item, Python simply ignores it. Create an empty set using the set() BIF, which is an example of a factory function: vaeCasmrrspiiegatanbtyleeits.eattno, eaawn,d distances = set() It is also possible to create and populate a set in one step. You can provide a list of data values between curly braces or specify an existing list as an argument to the set() BIF, which is the factory function: distances = {10.6, 11, 8, 10.6, "two", 7} iotAgfhnneoydrsadeutdupap.plliviceaadltueelissstainre Atighnnyeord“ejuadpm.liecCsao”toelli.sstinare distances = set(james) Factory Function: A factory function is used to make new data items of a particular type. For instance, “set()” is a factory function because it makes a new set. In the real world, factories make things, hence the name. 166 Chapter 5 comprehending data Tonight’s talk: Does list suffer from set envy? List: [sings] “Anything you can do, I can do better. I can do anything better than you.” Can you spell “d-a-t-a l-o-s-s”? Getting rid of data automatically sounds kinda dangerous to me. Seriously? And that’s all you do? And they pay you for that?!? Have you ever considered that I like my duplicate values. I’m very fond of them, you know. Which isn’t very often. And, anyway, I can always rely on the kindness of others to help me out with any duplicates that I don’t need. Set: I’m resisting the urge to say, “No, you can’t.” Instead, let me ask you: what about handling duplicates? When I see them, I throw them away automatically. But that’s what I’m supposed to do. Sets aren’t allowed duplicate values. Yes. That’s why I exist…to store sets of values. Which, when it’s needed, is a real lifesaver. That’s all I need to do. Very funny. You’re just being smug in an effort to hide from the fact that you can’t get rid of duplicates on your own. Yeah, right. Except when you don’t need them. I think you meant to say, “the kindness of set()”, didn’t you? Do this! To extract the data you need, replace all of that list iteration code in your current program with four calls to sorted(set(...))[0:3]. you are here 4 167 code review Head First Code Review The Head First Code Review Team has taken your code and annotated it in the only way they know how: they’ve scribbled all over it. Some of their comments are confirmations of what you might already know. Others are suggestions that might make your code better. Like all code reviews, these comments are an attempt to improve the quality of your code. I think we can make a few improvements here. def sanitize(time_string): if '-' in time_string: splitter = '-' elif ':' in time_string: A comment would be nice to have here. splitter = ':' else: return(time_string) (mins, secs) = time_string.split(splitter) return(mins + '.' + secs) Wiffhilaoetsneihsoamfppisetsnhinsegs?e!? with open('james.txt') as jaf: data = jaf.readline() james = data.strip().split(',') Wexhceerpet’sioynouhrandling with open('julie.txt') as data = juf.readline() juf: code? julie = data.strip().split(',') with open('mikey.txt') as mif: data = mif.readline() mikey = data.strip().split(',') Meet First the Head Code Review Team. Tcftdohuhauenetlrcdaetf’iusfofnaniaclc;ettbstio,oihtrnaesnoofs,ufiogtarndllitenuhyapgoecliuthcchanooteedfieoerdneyisonuthutloertordeata.toohYsimlasoenuatcleal ll athlete list. with open('sarah.txt') as saf: There’s a lot data = saf.readline() gbnouurointentdagewdtroeosniottfahhifnnearddrroedm,iiftt’tsyohoeu sarah = data.strip().split(',') print(sorted(set([sanitize(t) for print(sorted(set([sanitize(t) for print(sorted(set([sanitize(t) for print(sorted(set([sanitize(t) for t t t t in in in in james]))[0:3]) julie]))[0:3]) mikey]))[0:3]) sarah]))[0:3]) inside out. Ah, OK. We get it. The slice is applied to the list produced by “sorted()”, right? 168 Chapter 5 comprehending data Wfurnictteioynouhrernee.w Let’s take a few moments to implement the review team’s suggestion to turn those four with statements into a function. Here’s the code again. In the space provided, create a function to abstract the required functionality, and then provide one example of how you would call your new function in your code: with open('james.txt') as jaf: data = jaf.readline() james = data.strip().split(',') with open('julie.txt') as juf: data = juf.readline() julie = data.strip().split(',') with open('mikey.txt') as mif: data = mif.readline() mikey = data.strip().split(',') with open('sarah.txt') as saf: data = saf.readline() sarah = data.strip().split(',') Provide one example call. you are here 4 169 statement to function You were to take a few moments to implement the review team’s suggestion to turn those four with statements into a function. In the space provided, your were to create a function to abstract the required functionality, then provide one example of how you would call your new function in your code: with open('james.txt') as jaf: data = jaf.readline() james = data.strip().split(',') with open('julie.txt') as juf: data = juf.readline() julie = data.strip().split(',') with open('mikey.txt') as mif: data = mif.readline() mikey = data.strip().split(',') Create a new function. cAeoxddcdeep.tthioens-uhgagnedstliendg with open('sarah.txt') as saf: data = saf.readline() sarah = data.strip().split(',') def get_coach_data(filename): tAhcecespotleaarfgiluemnaenmte. as try: with open(filename) as f: Open read the the file, and data. data = f.readline() return(data.strip().split(‘,')) except IOError as ioerr: ttPhheeerfcdoaarltmliangtphrceioodsrpel.tito/rsettruiprntinrgickitotno Cisaslltinrgaigthhteffournwcatridon. print(‘File error: ' + str(ioerr)) return(None) T(tieoflliinytdouoicrcactuuseresrf)aaailbnuodruet.retthuerner“Nroorne” sarah = get_coach_data(‘sarah.txt') Provide process. the name of the file to 170 Chapter 5 comprehending data Test Drive It’s time for one last run of your program to confirm that your use of sets produces the same results as your list-iteration code. Take your code for a spin in IDLE and see what happens. Excellent! You’ve processed the coach’s data perfectly, while taking advantage of the sorted() BIF, sets, and list comprehensions. As you can imagine, you can apply these techniques to many different situations. You’re well on your way to becoming a Python data-munging master! LAcosodoeekxindpgeocegtsoetoddh, e!yobuursinlaetses.st That’s great work, and just what I need. Thanks! I’m looking forward to seeing you on the track soon... you are here 4 171 CHAPTER 5 python toolbox Your Python Toolbox You’ve got Chapter 5 under your belt and you’ve added some more Python techiques to your toolbox. Python Lingo •a•n“a•Idn“fncCd•“r-oftMooclp“rhtlmpoleFoeehailcelmtnueceltdenhencir”fc”rootetridtnigspseoiooohltnotCarnrtufocthtoreCiatifnrnsmnioh.isgngge.faihltni-ue-nthgnfi,”ocntttdtgar,-r”ispaoaapnnntr-plssseoipfefalrtsoioddeeorraaaismnmtddgasasian.tga. More Python Lingo i•toa•epAmpt“oL“rfssaielrsnidtcosefmtC”ooroa-mmulaspaisitrcnteicg.ohenesasnnosnmiiotoonern”reea-ttliihnosapenne)c(.oaifnsye cu•onnAotrad“isenerste”ndo-ddaautpacliociltaleetcmetssi.otnhoatf The sort() method changes the ordering of lists in-place. The sorted() BIF sorts most any data structure by providing copied sorting. Pass reverse=True to either sort() or sorted() to arrange your data in descending order. When you have code like this: new_l = [] for t in old_l: new_l. append(len(t)) rewrite it to use a list comprehension, like this: new_l = [len(t) for t in old_l] To access more than one data item from a list, use a slice. For example: my_list[3:6] accesses the items from index location 3 up-to-but-not-including index location 6. Create a set using the set() factory function. 172 Chapter 5 6 custom data objects Bundling code with data The object of my desire [sigh] is in a class of her own. It’s important to match your data structure choice to your data. And that choice can make a big difference to the complexity of your code. In Python, although really useful, lists and sets aren’t the only game in town. The Python dictionary lets you organize your data for speedy lookup by associating your data with names, not numbers. And when Python’s built-in data structures don’t quite cut it, the Python class statement lets you define your own. This chapter shows you how. this is a new chapter 173 additional data Coach Kelly is back (with a new file format) I love what you’ve done, but I can’t tell which line of data belongs to which athlete, so I’ve added some information to my data files to make it easy for you to figure it out. I hope this doesn’t mess things up much. The output from your last program in Chapter 5 was exactly what the coach was looking for, but for the fact that no one can tell which athlete belongs to which data. Coach Kelly thinks he has the solution: he’s added identification data to each of his data files: This is extra “sarah2.txt”, data added. with Sarah Sweeney,2002-6-17,2:58,2.58,2:39,2-25,2-55,2:54,2.18,2:55,2:55,2:22,2-21,2.22 Sarah’s full name Sarah’s date of birth Sarah’s timing data If you use the split() BIF to extract Sarah’s data into a list, the first data item is Sarah’s name, the second is her date of birth, and the rest is Sarah’s timing data. Do this! Let’s exploit this format and see how well things work. 174 Chapter 6 Grab the updated files from the Head First Python website. custom data objects Code Magnets Let’s look at the code to implement the strategy outlined at the bottom of the previous page. For now, let’s concentrate on Sarah’s data. Rearrange the code magnets at the bottom of this page to implement the list processing required to extract and process Sarah’s three fastest times from Coach Kelly’s raw data. Hint: the pop() method removes and returns a data item from the specified list location. Rearrange the magnets here. def sanitize(time_string): if '-' in time_string: splitter = '-' Tithwea“ssainnitCihzaep()t”erfu5n.ction is as elif ':' in time_string: splitter = ':' else: return(time_string) (mins, secs) = time_string.split(splitter) return(mins + '.' + secs) def get_coach_data(filename): try: The also “fgreotm_ctohaechla_stdacthaa(p)t”efru. nction is with open(filename) as f: data = f.readline() return(data.strip().split(',')) except IOError as ioerr: print('File error: ' + str(ioerr)) return(None) sarah (sarah_name, sarah_dob) "'s fastest times are: " + = print(sarah_name + get_coach_data('sarah2.txt') = sarah.pop(0), sarah.pop(0) str(sorted(set([sanitize(t) for t in sarah]))[0:3])) you are here 4 175 sarah’s times Code Magnets Solution Let’s look at the code to implement the strategy outlined earlier. For now, let’s concentrate on Sarah’s data. You were to rearrange the code magnets at the bottom of the previous page to implement the list processing required to extract and process Sarah’s three fastest times from Coach Kelly’s raw data. def sanitize(time_string): if '-' in time_string: splitter = '-' elif ':' in time_string: splitter = ':' else: return(time_string) (mins, secs) = time_string.split(splitter) return(mins + '.' + secs) def get_coach_data(filename): try: with open(filename) as f: data = f.readline() return(data.strip().split(',')) except IOError as ioerr: print('File error: ' + str(ioerr)) return(None) “asSUanarsdreaahtth”h’hseevnadfraaiutasnsabcilgtefni.iolientitntotoottuahrenlist, sarah = get_coach_data('sarah2.txt') (sarah_name, sarah_dob) = sarah.pop(0), sarah.pop(0) The “pop(0)” call print(sarah_name + "'s fastest times are: " + “rrtltpvetiehsaohtmhtelpeuue.o(.remfv0fnsTreis)owstra”nsoaondttrndatcedttohamawafelsolossvfniaegtdranoomasmtead str(sorted(set([sanitize(t) for t in sarah]))[0:3])) attAfohtecdeuicrssta.polllamytomte“hspesrairngetes(uw)lt”itsihsyinouus’erde variables. 176 Chapter 6 Test Drive Let’s run this code in IDLE and see what happens. custom data objects Your latest code Tiusnhmdiseuorcsuhttampnuodtraeble. This program works as expected, and is fine…except that you have to name and create Sarah’s three variables in such as way that it’s possible to identify which name, date of birth, and timing data relate to Sarah. And if you add code to process the data for James, Julie, and Mikey, you’ll be up to 12 variables that need juggling. This just about works for now with four athletes. But what if there are 40, 400, or 4,000 athletes to process? Although the data is related in “real life,” within your code things are disjointed, because the three related pieces of data representing Sarah are stored in three separate variables. you are here 4 177 keys and values Use a dictionary to associate data Lists are great, but they are not always the best data structure for every situation. Let’s take another look at Sarah’s data: Sarah’s full name Sarah’s date of birth Sarah’s timing data Sarah Sweeney,2002-6-17,2:58,2.58,2:39,2-25,2-55,2:54,2.18,2:55,2:55,2:22,2-21,2.22 There’s a definite structure here: the athlete’s name, the date of birth, and then the list of times. Let’s continue to use a list for the timing data, because that still makes sense. But let’s make the timing data part of another data structure, which associates all the data for an athlete with a single variable. We’ll use a Python dictionary, which associates data values with keys: The “keys” Name DOB Times "Sarah Sweeney" "2002-6-17" Tashethaess“ovcailauteesd” data, also known [2:58,2.58,2:39,2-25,2-55,2:54,2.18,2:55,2:55,2:22,2-21,2.22] Dictionary A built-in data structure (included with Python) that allows you to associate data with keys, as opposed to numbers. This lets your in-memory data closely match the structure of your actual data. 178 Chapter 6 custom data objects Tonight’s talk: To use a list or not to use a list? Dictionary: Hi there, List. I hear you’re great, but not always the best option for complex data. That’s where I come in. True. But when you do, you lose any structure associated with the data you are processing. Isn’t it always? You guess so? When it comes to modeling your data in code, it’s best not to guess. Be firm. Be strong. Be assertive. Use a dictionary. [laughs] Oh, I do love your humor, List, even when you know you’re on thin ice. Look, the rule is simple: if your data has structure, use a dictionary, not a list. How hard is that? Which rarely makes sense. Knowing when to use a list and when to use a dictionary is what separates the good programmers from the great ones, right? Geek Bits List: What?!? Haven’t you heard? You can put anything into a list, anything at all. Well…assuming, of course, that structure is important to you. Ummm, uh…I guess so. That sounds like a slogan from one of those awful self-help conferences. Is that where you heard it? Not that hard, really. Unless, of course, you are a list, and you miss being used for every piece of data in a program… I guess so. Man, I do hate it when you’re right! The Python dictionary is known by different names in other programming languages. If you hear other programmers talking about a “mapping,” a “hash,” or an “associative array,” they are talking about a “dictionary.” you are here 4 179 idle session Let’s see the Python dictionary in action. Follow along with this IDLE session on your computer, ensuring that you get the same results as shown. Start by creating two empty dictionaries, one using curly braces and the other using a factory function: >>> cleese = {} >>> palin = dict() >>> type(cleese) >>> type(palin) caBonontefhmirpmtteeycdh.dniicqtuieosnacrrye,ataes Add some data to both of these dictionaries by associating values with keys. Note the actual structure of the data is presenting itself here, as each dictionary has a Name and a list of Occupations. Note also that the palin dictionary is being created at the same time: >>> cleese['Name'] = 'John Cleese' >>> cleese['Occupations'] = ['actor', 'comedian', 'writer', 'film producer'] >>> palin = {'Name': 'Michael Palin', 'Occupations': ['comedian', 'actor', 'writer', 'tv']} With your data associated with keys (which are strings, in this case), it is possible to access an individual data item using a notation similar to that used with lists: Use square brackets to index into the dictionary to access >>> palin['Name'] data items, but instead of numbers, index with keys. 'Michael Palin' >>> cleese['Occupations'][-1] 'film producer' TUithseeimnknouofmfbtethrhesisltisoatsaa“ciscnsedoscesixaa-teclidhstawinitiitnehgm”Osactncodurperadetaiaodtnsfa…rop”.martriicguhltartodilcetfiotn: a“r…ytkheeyl.ast As with lists, a Python dictionary can grow dynamically to store additional key/value pairings. Let’s add some data about birthplace to each dictionary: >>> palin['Birthplace'] = "Broomhill, Sheffield, England" Provide the data associated with the new key. >>> cleese['Birthplace'] = "Weston-super-Mare, North Somerset, England" Unlike lists, a Python dictionary does not maintain insertion order, which can result in some unexpected behavior. The key point is that the dictionary maintains the associations, not the ordering: >>> palin {'Birthplace': 'Broomhill, Sheffield, England', 'Name': 'Michael Palin', 'Occupations': ['comedian', 'actor', 'writer', 'tv']} >>> cleese {'Birthplace': 'Weston-super-Mare, North Somerset, England', 'Name': 'John Cleese', 'Occupations': ['actor', 'comedian', 'writer', 'film producer']} The was ordering inserted. maintained by Python is different Don’t worry about it; this is OK. from how the data 180 Chapter 6 custom data objects It’s time to apply what you now know about Python’s dictionary to your code. Let’s continue to concentrate on Sarah’s data for now. Strike out the code that you no longer need and replace it with new code that uses a dictionary to hold and process Sarah’s data. def sanitize(time_string): if '-' in time_string: splitter = '-' elif ':' in time_string: splitter = ':' else: return(time_string) (mins, secs) = time_string.split(splitter) return(mins + '.' + secs) def get_coach_data(filename): try: with open(filename) as f: data = f.readline() return(data.strip().split(',')) except IOError as ioerr: Strike you no loountgetrhneeceodd. e print('File error: ' + str(ioerr)) return(None) sarah = get_coach_data('sarah2.txt') (sarah_name, sarah_dob) = sarah.pop(0), sarah.pop(0) print(sarah_name + "'s fastest times are: " + Add your using and dictionary processing code here. str(sorted(set([sanitize(t) for t in sarah]))[0:3])) you are here 4 181 dictionary data It’s time to apply what you now know about Python’s dictionary to your code. Let’s continue to concentrate on Sarah’s data for now. You were to strike out the code that you no longer needed and replace it with new code that uses a dictionary to hold and process Sarah’s data. def sanitize(time_string): if '-' in time_string: splitter = '-' elif ':' in time_string: splitter = ':' else: return(time_string) (mins, secs) = time_string.split(splitter) return(mins + '.' + secs) def get_coach_data(filename): try: with open(filename) as f: data = f.readline() return(data.strip().split(',')) except IOError as ioerr: print('File error: ' + str(ioerr)) return(None) Ycooduedaonny’tmonreee.d this sarah = get_coach_data('sarah2.txt') (sarah_name, sarah_dob) = sarah.pop(0), sarah.pop(0) print(sarah_name + "'s fastest times are: " + Create an empty str(sorted(set([sanitize(t) for t in sarah]))[0:3])) dictionary. sarah_data = {} sarah_data[‘Name’] = sarah.pop(0) sarah_data[‘DOB’] = sarah.pop(0) sarah_data[‘Times’] = sarah taPhsosepoucdliaiacttteiinotgnhaterhydeikcdetayitos.an.afrryowmitthhethfeiledawtiathby print(sarah_data[‘Name’] + “’s fastest times are: “ + pRreofceersstinogtthheeddicattstiaor.n(asroyrtwehde(nset([sanitize(t) for t in sarah_data[‘Times’]]))[0:3])) 182 Chapter 6 custom data objects Test Drive Let’s confirm that this new version of your code works exactly as before by testing your code within the IDLE environment. aYpsroouedraurdcliieecsrt.itohnearsyamceodreesults Which, again, works as expected…the difference being that you can now more easily determine and control which identification data associates with which timing data, because they are stored in a single dictionary. Although, to be honest, it does take more code, which is a bit of a bummer. Sometimes the extra code is worth it, and sometimes it isn’t. In this case, it most likely is. Let’s review your code to see if we can improve anything. you are here 4 183 code review Head First Code Review The Head First Code Review Team has been at it again: they’ve scribbled all over your code. Some of their comments are confirmations; others are suggestions. Like all code reviews, these comments are an attempt to improve the quality of your code. def sanitize(time_string): if '-' in time_string: splitter = '-' elif ':' in time_string: splitter = ':' else: return(time_string) (mins, secs) = time_string.split(splitter) return(mins + '.' + secs) It’s great to see you taking some of our suggestions on board. Here are a few more... def get_coach_data(filename): try: with open(filename) as f: data = f.readline() return(data.strip().split(',')) except IOError as ioerr: print('File error: ' + str(ioerr)) return(None) sarah = get_coach_data('sarah2.txt') sarah_data = {} sarah_data['Name'] = sarah.pop(0) sarah_data['DOB'] = sarah.pop(0) sarah_data['Times'] = sarah Rattfplahhouotiinnpsshgcuest,pliarirtwotoutnhecayhdetaasinndsnodoininctb,gthuidiatiwolvdoniemtianihtirtggiyhnhatetlathlsfheeiunevodnepogcnipctneotmteisoi_eoangdnkcoareo?treaosytcIenuhnaar_sfsneldaiyscaattotto.u,a(dign)oo Then, all you need to dictionary from the appropriate function do is create the data file using an call, right? print(sarah_data['Name'] + "'s fastest times are: " + str(sorted(set([sanitize(t) for t in sarah_data['Times']]))[0:3])) You might want to consider moving this code into the get_coach_data() would rather nicely abstract away these processing details. But whether your code, after all! function, you do or too, not because is up to doing so you. It’s 184 Chapter 6 custom data objects Actually, those review comments are really useful. Let’s take the time to apply them to your code. There are four suggestions that you need to adjust your code to support: 1. Create the dictionary all in one go. 2. Move the dictionary creation code into the get_coach_data() function, returning a dictionary as opposed to a list. 3. Move the code that determines the top three times for each athlete into the get_coach_data() function. 4. Adjust the invocations within the main code to the new version of the get_coach_data() function to support it’s new mode of operation. Grab your pencil and write your new get_coach_data() function in the space provided below. Provide the four calls that you’d make to process the data for each of the athletes and provide four amended print() statements: you are here 4 185 reviews are in You were to take the time to apply the code review comments to your code. There were four suggestions that you needed to adjust your code to support: 1. Create the dictionary all in one go. 2. Move the dictionary creation code into the get_coach_data() function, returning a dictionary as opposed to a list. 3. Move the code that determines the top three times for each athlete into the get_coach_data() function. 4. Adjust the invocations within the main code to the new version of the get_coach_data() function to support its new mode of operation. You were to grab your pencil and write your new get_coach_data() function in the space provided below, as well as provide the four calls that you’d make to process the data for each of the athletes and provide four amended print() statements: def get_coach_data(filename): try: with open(filename) as f: data = f.readline() 1lBdi.siEtCctFrtieOooanRthaEeorlyadcrattellehamtienipndoogarntaetarhygeo. templ = data.strip().split(‘,’) return({‘Name’ : templ.pop(0), ‘DOB’ : templ.pop(0), 2no.wTphaerdticotfiotnhaeryfucnrcetatioino.n code is ‘Times’: str(sorted(set([sanitize(t) for t in templ]))[0:3])}) except IOError as ioerr: print(‘File error: ‘ + str(ioerr)) return(None) 3t.opTthehrceoedsecotrheastisdeptarertmoinfesthtehe function, too. 4fasotd.rajCtuasaetnlml eattnthhtheelea“ftpsuernnicneattenid(do)en”d. james = get_coach_data(‘james2.txt’) fcWooerdeatrhfeeorsohtoohnweeirnagttohhnrleleyteetish(beaesectartuiwsveioalrlieenpxeeesartocifinsge)i.t print(james[‘Name’] + “’s fastest times are: “ + james[‘Times’]) 186 Chapter 6 custom data objects Test Drive Let’s confirm that all of the re-factoring suggestions from the Head First Code Review Team are working as expected. Load your code into IDLE and take it for a spin. All of moved the into dtahteafpurnoccteiossni.ng is Tnwohitwishdctoishdpeeliaryhsatsitmbheeese.nnacmoensoidfetrahbelyatthildeiteed up and associated Looking good! To process additional athletes, all you need is two lines of code: the first invokes the get_coach_data() function and the second invokes print(). And if you require additional functionality, it’s no big deal to write more functions to provide the required functionality, is it? you are here 4 187 associate custom code with custom data Wait a minute...you’re using a dictionary to keep your data all in one place, but now you’re proposing to write a bunch of custom functions that work on your data but aren’t associated with it. Does that really make sense? Keeping your code and its data together is good. It does indeed make sense to try and associate the functions with the data they are meant to work on, doesn’t it? After all, the functions are only going to make sense when related to the data—that is, the functions will be specific to the data, not general purpose. Because this is the case, it’s a great idea to try and bundle the code with its data. But how? Is there an easy way to associate custom code, in the form of functions, with your custom data? 188 Chapter 6 Bundle your code and its data in a class Like the majority of other modern programming languages, Python lets you create and define an object-oriented class that can be used to associate code with the data that it operates on. custom data objects Why would anyone want to do this? Using a class helps reduce complexity. By associating your code with the data it works on, you reduce complexity as your code base grows. So what’s the big deal with that? Reduced complexity means fewer bugs. Reducing complexity results in fewer bugs in your code. However, it’s a fact of life that your programs will have functionality added over time, which will result in additional complexity. Using classes to manage this complexity is a very good thing. Yeah? But...who really cares? Fewer bugs means more maintainable code. Using classes lets you keep your code and your data together in one place, and as your code base grows, this really can make quite a difference. Especially when it’s 4 AM and you’re under a deadline… you are here 4 189 get some class Define a class Python follows the standard object-oriented programming model of providing a means for you to define the code and the data it works on as a class. Once this definition is in place, you can use it to create (or instantiate) data objects, which inherit their characteristics from your class. Within the object-oriented world, your code is often referred to as the class’s methods, and your data is often referred to as its attributes. Instantiated data objects are often referred to as instances. The “raw” data "Sarah Sweeney","2002-6-17",[2:58,2.58,2:39,2-25,2-55,2:54,2.18,2:55,2:55,2:22,2-21,2.22] The Object Factory The factory primed with has been your class. Hwceohrdiceehaaarnredeyipotausrcakisansgsotecdaiantttoeiadtcoeddnattoaabi.njeycotusr, Mikey’s object instance Each object is created from the class and shares a similar set of characteristics. The methods (your code) are the same in each instance, but each object’s attributes (your data) differ because they were created from your raw data. Let’s look at how classes are defined in Python. Julie’s object instance 190 Chapter 6 James’s object instance Use class to define classes Python uses class to create objects. Every defined class has a special method called __init__(), which allows you to control how objects are initialized. Methods within your class are defined in much the same way as functions, that is, using def. Here’s the basic form: custom data objects The the keyword starts definition. dGeivsecryipotuirveclnaassmea. nice, class Athlete: Don’t forget the colon! def __init__(self): # The code to initialize a "Athlete" object. ... Tanhdata’fs taerdotuhbelewuonrdder“sicnoitr”e. before Tobhjeecctodgeoetshianthienritei.alizes each Creating object instances With the class in place, it’s easy to create object instances. Simply assign a call to the class name to each of your variables. In this way, the class (together with the __init__() method) provides a mechanism that lets you create a custom factory function that you can use to create as many object instances as you require: a = Athlete() The brackets wcrheicahteisatnheewn variable. a“tAsestliglhnPleeydtteht”ooonbatjeoct, All of unique tahnedsearvear“ioafbletsypaer”e Athlete. b = Athlete() c = Athlete() d = Athlete() Unlike in C++-inspired languages, Python has no notion of defining a constructor called “new.” Python does object contruction for you, and then lets you customize your object’s initial state using the __init__() method. you are here 4 191 note to self The importance of self To confirm: when you define a class you are, in effect, defining a custom factory function that you can then use in your code to create instances: Tahreeftearregnecte itdoenytoiufrieirnstthaantceholds a = Athlete() When Python processes this line of code, it turns the factory function call into the following call, which identifies the class, the method (which is automatically set to __init__()), and the object instance being operated on: Invoke the function. class’s custom factory The name of the class Athlete().__init__(a) Tofhetthaerogbejtecidteinntstifainecre The name of the method Now take another look at how the __init__() method was defined in the class: def __init__(self): # The code to initialize an "Athlete" object. ... Check out what Python turns your object creation invocation into. Notice anything? The target identifer is assigned to the self argument. This is a very important argument assignment. Without it, the Python interpreter can’t work out which object instance to apply the method invocation to. Note that the class code is designed to be shared among all of the object instances: the methods are shared, the attributes are not. The self argument helps identify which object instance’s data to work on. 192 Chapter 6 Every method’s first argument is self In fact, not only does the __init__() method require self as its first argument, but so does every other method defined within your class. Python arranges for the first argument of every method to be the invoking (or calling) object instance. Let’s extend the sample class to store a value in a object attribute called thing with the value set during initialization. Another method, called how_big(), returns the length of thing due to the use of the len() BIF: custom data objects Tatcsohaselilage“ndcisnla“iatss”esslufacpto.tptdlhireeiindbnguo”vtw.aelue class Athlete: def __init__(self, value=0): self.thing = value def how_big(self): return(len(self.thing)) Tlehnget“hhoowf_“bsiegl(f).”thminegt”h.od returns the When you invoke a class method on an object instance, Python arranges for the first argument to be the invoking object instance, which is always assigned to each method’s self argument. This fact alone explains why self is so important and also why self needs to be the first argument to every object method you write: “itNsnhesoltefta”ecnacttlelohi.neigdueosnbetjeoifcfyt What you write: What Python executes: d = Athlete("Holy Grail") d.how_big() Athlete.__init__(d, "Holy Grail") The class The method Tinhdeenttairfgeert(or instance) Athlete.how_big(d) you are here 4 193 idle session Let’s use IDLE to create some object instances from a new class that you’ll define. Start by creating a small class called Athlete: >>> class Athlete: tNwooteofthtehedeafraguulmtevnatlus.es for def __init__(self, a_name, a_dob=None, a_times=[]): self.name = a_name self.dob = a_dob self.times = a_times Tathtrereibuattetsriubsuitngestahreesiunpitpilaieldizeadrgaunmdenatssdigantead. to three class With the class defined, create two unique object instances which derive their characteristcs from the Athlete class: >>> sarah = Athlete('Sarah Sweeney', '2002-6-17', ['2:58', '2.58', '1.56']) >>> james = Athlete('James Jones') >>> type(sarah) >>> type(james) “Cjaomnfesir”martehaatthbleottehs.“sarah” and Create “james” values). two using utnhiqeudeeaftahullettaesrg(uwmitehnt Even though sarah and james are both athletes and were created by the Athlete class’s factory function, they are stored at different memory addreses: >>> sarah <__main__.Athlete object at 0x14d23f0> >>> james <__main__.Athlete object at 0x14cb7d0> Tdthhifeefseemreamfreroortmyheatdhmdeermveasolsrueyfsoarrde“dpsroaerrstaseehds” ooannndoyuo“rujarcsmo.emTsp”huedtiekfref,yewrp.hoiicnhtwisill Now that sarah and james exist as object instances, you can use the familiar dot notation to access the attributes associated with each: >>> sarah.name 'Sarah Sweeney' >>> james.name 'James Jones' >>> sarah.dob '2002-6-17' >>> james.dob >>> sarah.times “Tdohbe”“, jsaomneos”thoibnjgecatppienastrsanocne has no screen. value for ['2:58', '2.58', '1.56'] >>> james.times [] 194 Chapter 6 custom data objects Here’s your code (except for the santize() function, which doesn’t need to change). With your pencil, write code to define the Athlete class. In addition to the __init__() method, define a new method called top3() that, when invoked, returns the top three times. Be sure to adjust the get_coach_data() function to return an Athlete object as opposed to a dictionary, and don’t forget to amend your print() statements, too. cWlarsistecoydoeurheArteh. lete def get_coach_data(filename): try: with open(filename) as f: data = f.readline() templ = data.strip().split(',') return({'Name' : templ.pop(0), WtahshisaotpfpunonesceetddisotntooraecthduaircnntgsieoanhnaerrAyet?htloeteensoubrjeect 'DOB' : templ.pop(0), 'Times': str(sorted(set([sanitize(t) for t in templ]))[0:3])}) except IOError as ioerr: print('File error: ' + str(ioerr)) return(None) james = get_coach_data('james2.txt') This line needs to of code change, too. print(james['Name'] + "'s fastest times are: " + james['Times']) you are here 4 195 class athlete Here’s your code (except for the santize() function, which doesn’t need to change). With your pencil, you were to write code to define the Athlete class. In addition to the __init__() method, you were to define a new method called top3() that, when invoked, returns the top three times. You were to be sure to adjust the get_coach_data() function to return an Athlete object as opposed to a dictionary, and you weren’t to forget to amend print(), too. class Athlete: def __init__(self, a_name, a_dob=None, a_times=[]): self.name = a_name self.dob = a_dob self.times = a_times def top3(self): Did you remember to use “self”? return(sorted(set([sanitize(t) for t in self.times]))[0:3]) Tmcoohdsetereri’ssectneaonkttehniInDsgtLrnEaewigshehstesirfoenr.oams this the def get_coach_data(filename): try: with open(filename) as f: ocRboejdmeecotvaencdrtehraeetpidolainccteciooitdnaewriyintshctreAeaatdth.iloente data = f.readline() templ = data.strip().split(',') return({'Name' : templ.pop(0), 'DOB' : templ.pop(0), Athlete(templ.pop(0), templ.pop(0), templ) 'Times': str(sorted(set([sanitize(t) for t in templ]))[0:3])}) except IOError as ioerr: print('File error: ' + str(ioerr)) return(None) james = get_coach_data('james2.txt') james.name Use the at your dot notation data. to get str(james.top3()) trmIoneevstiuotlhktsoesddttishaopenlaad“ytsctooorpnni3nvs(egc)r”rptereiitno.rs print(james['Name'] + "'s fastest times are: " + james['Times']) 196 Chapter 6 custom data objects Test Drive With these changes applied to your program, let’s ensure you continue to get the same results as earlier. Load your code into IDLE and run it. Tpnrohotegrschaoomdw.enthoerteh,eb“ustanititiizs es(t)i”ll function part of is this Cool! There’s no change here. And to make objects do more, I just add more methods, right? Yes, that’s correct: more functionality = more methods. Simply add methods to encapsulate the new functionality you need within your class. There’s no limit to how many methods a class can have, so feel free to knock yourself out! you are here 4 197 no dumb questions Q: I’m not sure I see why the top3() method is coded to return a three-item list, as opposed to a string? Surely a string would make the print() statement in the main program easier to write? A: It would, but it wouldn’t be as flexible. By returning a list (albeit a small one), the top3() method lets the calling code decide what happens next, as opposed to forcing the caller to work with a string. Granted, the current program needs to treat the list like a string, but not all programs will want or need to. Q: Why does the class even need the top3() method? Why not store the top three times as an attribute within the class and create it as part of the object’s creation? A: Again, better not to, because doing so is less flexible. If you compute and store the top three times at object creation, you make it harder to extend the list of timing data associated with the object. For instance, if you add more timing data after the object is created, you’ll need to arrange to recompute the top three (because the new times might be fast) and update the attribute. However, when you compute the top three times “on the fly” using a call to the top3() method, you always ensure you’re using the most up-to-date data. Q: OK, I get that. But, with a little extra work, I could do it during object creation, right? A: Well, yes…but we really don’t advise that. By preserving the original data in each object’s attributes, you are supporting the extension of the class to support additional requirements in the future (whatever they might be). If you process the data as part of the object initialization code, the assumptions you make about how programmers will use your class might just come back to bite you. Q: But what if I’m the only programmer that’ll ever use a custom class that I write? A: Trust us: you’ll thank yourself for coding your class to be as flexible as possible when you come to use it for some other purpose in a future project. When you are creating a class, you have no idea how it will be used by other programmers in their projects. And, if you think about, you have no idea how you might use it in the future, too. Q: OK, I think I’m convinced. But tell me: how do I go about adding more times to my existing Athlete objects? A: To do more, add more methods. With your Athlete class created, it’s a breeze to extend it to do more work for you: simply add more methods. So, if you want to add a single new timing value to your times attribute, define a method called add_time() to do it for you. Additionally, you can add a list of times by defining a method called add_times().Then all you need to do in your code is say something like this: sarah.add_time('1.31') to add a single time to Sarah’s timing data, or say this: james.add_times(['1.21','2.22']) to add a bunch of times to James’s data. Q: But surely, knowing that times is a list, I could write code like this to do the same thing? sarah.times.append('1.31') james.times.append(['1.21','2.22']) A: You could, but that would be a disaster. Q: What?!? Why do you say that? There’s nothing wrong with my suggestion, is there? A: Well…it does indeed work. However, the problem with writing code like that is that it exposes (and exploits) that fact that the timing data is stored in a list within the Athlete class. If you later change your class implementation to use (for instance) a string instead of a list, you may well break all of the existing code that uses your class and that exploits the fact that the timing data is a list. By defining your own API with add_time() and add_ times(), you leave open the possibility that the way the data is stored within your class can change in the future (obviously, only if such a change makes sense). It is worth noting that one of the reasons for using object orientation is to hide away the details of a class’s implementation from the users of that class. Defining your own API directly supports this design ideal. Exposing the internals of your class’s implementation and expecting programmers to exploit it breaks this fundamental ideal in a very big way. 198 Chapter 6 custom data objects Amdedthyoodusrhneerwe. Let’s add two methods to your class. The first, called add_time(), appends a single additional timing value to an athlete’s timing data. The second, add_times(), extends an athlete’s timing data with one or more timing values supplied as a list. Here’s your current class: add the code to implement these two new methods. class Athlete: def __init__(self, a_name, a_dob=None, a_times=[]): self.name = a_name self.dob = a_dob self.times = a_times def top3(self): return(sorted(set([sanitize(t) for t in self.times]))[0:3]) Don’t put down the pencil just yet! Provide a few lines of code to test your new functionality: you are here 4 199 more methods Let’s add two methods to your class. The first, called add_time(), appends a single additional timing value to an athlete’s timing data. The second, add_times(), extends an athlete’s timing data with one of more timing values supplied as a list. Here’s your current class: you were to add the code to implement these two new methods. class Athlete: def __init__(self, a_name, a_dob=None, a_times=[]): self.name = a_name self.dob = a_dob self.times = a_times def top3(self): return(sorted(set([sanitize(t) for t in self.times]))[0:3]) Don’t forget to use “self”!!! def add_time(self, time_value): self.times.append(time_value) def add_times(self, list_of_times): self.times.extend(list_of_times) tTaiapmkpieenngtdhvieatlusteuosp.ptliheed argument and existing list of Tavaankldueeestxhwteeitnlhidsttthohefemse.uxpisptliiendg arguments list of timing Create object a new instance for Vera. 200 Chapter 6 While still holding on firmly to your pencil, you were to provide a few lines of code to test your new functionality: vera = Athlete(‘Vera Vi’) vera.add_time(‘1.31’) Add a single timing value. print(vera.top3()) This will display a list with only one value in it: 1.31. vera.add_times([‘2.22’, “1-21”, ‘2:22’]) Add three more timing values. print(vera.top3()) The top 3 timing scores are now: 1.21, 1.31 and 2.22. Do this! custom data objects Amend your code with the updated version of your Athlete class before proceeding with this Test Drive. Test Drive After running your existing program, try out your test code in the IDLE shell to confirm that everything is working as expected. As expected. Create a new athlete. Add one timing value. Display the top three times (there’s only one, so that’s all you see). Add three more timing values. Display the top three times (which, now, makes a little more sense). Great: it worked. You’ve packaged your code with your data and created a custom class from which you can create objects that share behaviors. And when extra functionality is required, add more methods to implement the required functionality. By encapsulating your athlete code and data within a custom class, you’ve created a much more maintainable piece of software. You will thank yourself for doing this when, in six months, you need to amend your code. Well done. This is really coming along! you are here 4 201 reinventing the wheel Emmm...maybe I’m missing something, but isn’t your Athlete class wasteful? I mean, you’ve extended it with functionality that’s already in lists, which feels a little like reinventing the wheel to me... Yes, your Athlete class is much like a list. Your Athlete class does indeed behave like a list most of the time, and you’ve added methods to expose some list functionality to the users of your class. But it’s true: you are reinventing the wheel here. Your add_time() method is a thin wrapper around the list append() method and your add_times() method is list’s extend() method in disguise. In fact, your Athlete class only differs from Python’s list due to the inclusion of the name and dob object attributes. 202 Chapter 6 custom data objects Wouldn't it be dreamy if there were a way to extend a built-in class with custom attributes? But I know it's just a fantasy… you are here 4 203 inherit class Inherit from Python’s built-in list Python’s class lets you create a custom class from scratch, just like you did with your Athlete class. However, class also lets you create a class by inheriting from any other existing class, including Python’s built-in data structure classes that provide you with list, set, and dict. Such classes are referred to as subclasses. What’s really nice is that when you inherit from an existing class (such as list), you are given all of the existing functionality for free. As your existing class is really nothing more than a list with added attributes, perhaps a better design is to kill off your Athlete class and replace it with a class that inherits from the built-in list class? It’s certainly worth considering, isn’t it? Sorry to hear about your Athlete class. But, according to my files, you’re in line to inherit a mountain of functionality from the built-in list class. Congratulations, you’re rich! 204 Chapter 6 Slippery lawyer-type custom data objects Tonight’s talk: Inheritance, a.k.a. He looks just like his father. Custom Class: Programmers like me because they get to control everything in their code…and you know programmers: they love to code. Design! Phooey! Real programmers eat, sleep, dream, snore, and exhale code. All that design talk is for people who can’t code! No, no, no: you’re not listening. It’s all done with control. When you build everything from the ground up, you’re in control, as it’s all your code. Of course, especially when there are custom requirements to be taken into consideration. In that case, a brand-spanking new custom class is the only way to go. Yeah, right…it’s a win-win for you, not me. I guess so, although I’m still a fan of custom code… Inherited Class: Yes, they do. But sometimes writing everything from scratch is not the best design decision. Is it really? So, you’re saying it’s much better to do everything from scratch and repeat the work of others, because your way is the best way. Are you serious?!? And you’re happy to reinvent the wheel, even though someone else solved that problem eons ago? Not if you can extend someone else’s class to handle your custom requirements. That way, you get the best of both worlds: inheritied functionality (so you’re not reinventing the wheel) together with the custom bits. It’s a win-win situation. But it’s not about us: it’s to do with making the life of the programmer easier, even the ones that live to code, right? you are here 4 205 idle session Let’s see what’s involved in inheriting from Python’s built-in list class. Working in IDLE’s shell, start by creating a custom list derived from the built-in list class that also has an attribute called name: >>> class NamedList(list): def __init__(self, a_name): list.__init__([]) self.name = a_name Provide the class derives name of from. the class that this new Iansistigianliztehethaergdumereinvetdtfortohme class, and then attribute. With your NamedList class defined, use it to create an object instance, check the object’s type (using the type() BIF), and see what it provides (using the dir() BIF): >>> johnny = NamedList("John Paul Jones") >>> type(johnny) Create a new “NamedList” object instance. Yes, “johnny” is a “NamedList”. >>> dir(johnny) ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dict__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__module__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'append', 'count', 'extend', 'index', 'insert', 'name', 'pop', 'remove', 'reverse', 'sort'] “johnny” can do everything a list can, as well as store data in the “name” attribute. Use some of the functionality supplied by the list class to add to the data stored in johnny: >>> johnny.append("Bass Player") >>> johnny.extend(['Composer', "Arranger", "Musician"]) >>> johnny ['Bass Player', 'Composer', 'Arranger', 'Musician'] >>> johnny.name 'John Paul Jones' Access the list data, the attribute data. as well as Amdedthdoadtsaprtoovtidheed“bNyamthedeLliisstt”buuislitngint. he Because johnny is a list, it’s quite OK to do list-type things to it: >>> for attr in johnny: print(johnny.name + " is a " + attr + ".") “juosehninty”whiserliekveeranyyouo’tdhuesreliastl,istso. feel free to John Paul Jones is a Bass Player. John Paul Jones is a Composer. John Paul Jones is a Arranger. John Paul Jones is a Musician. 206 Chapter 6 Confirmation: John’s a busy boy. § custom data objects Here is the code for the now defunct Athlete class. In the space provided below, rewrite this class to inherit from the built-in list class. Call your new class AthleteList. Provide a few lines of code to exercise your new class, too: class Athlete: def __init__(self, a_name, a_dob=None, a_times=[]): self.name = a_name self.dob = a_dob self.times = a_times def top3(self): return(sorted(set([sanitize(t) for t in self.times]))[0:3]) def add_time(self, time_value): self.times.append(time_value) Wclarsitsecoydouerhneerwe. def add_times(self, list_of_times): self.times.extend(list_of_times) cEoxdeerchieserey. our you are here 4 207 new athletelist Here is the code for the now defunct Athlete class. In the space provided below, you were to rewrite this class to inherit from the built-in list class. You were to call your new class AthleteList, as well as provide a few lines of code to exercise your new class: class Athlete: def __init__(self, a_name, a_dob=None, a_times=[]): self.name = a_name self.dob = a_dob self.times = a_times def top3(self): return(sorted(set([sanitize(t) for t in self.times]))[0:3]) The class name has changed. def add_time(self, time_value): self.times.append(time_value) def add_times(self, list_of_times): self.times.extend(list_of_times) These methods aren’t needed anymore. class AthleteList(list): Inherit from the built-in list class. def __init__(self, a_name, a_dob=None, a_times=[]): “NstNihmaoimtislahecrdionLdgteoisnteits”whvei nehiretyrec… ode. list.__init__([]) self.name = a_name self.dob = a_dob self.extend(a_times) def top3(self): The data itself is the timing data, astottrhibeut“teimises” gone. Use the new class’s name. return(sorted(set([sanitize(t) for t in self]))[0:3]) vera = AthleteList(‘Vera Vi’) Nitymnohoheuewetrchbtiatouhnidilnatstgu-stfeyionoriuotgl’imesrstte, your work on. vera.append(‘1.31’) print(vera.top3()) vera.extend([‘2.22’, “1-21”, ‘2:22’]) print(vera.top3()) This code exercising does your a good job new class. of 208 Chapter 6 Do this! In your code, replace your Athlete class code with your new AthleteList class code, and don’t forget to change get_coach_data() to return an AthleteList object instance as opposed to an Athlete object instance. custom data objects Q: Sorry…but not three minutes ago you were telling me not to expose the inner workings of my class to its users, because that was fundamentally a bad idea. Now you’re doing the exact opposite! What gives? A: Well spotted. In this particular case, it’s OK to expose the fact that the class is built on top of list. This is due to the fact that the class is deliberately called AthleteList to distinguish it from the more generic Athlete class. When programmers see the word “list” in a class name, they are likely to expect the class to work like a list and then some. This is the case with AthleteList. Q: And I can inherit from any of the built-in types? A: Yes. Q: What about inheriting from more than one class…does Python support multiple interitance? A: Yes, but it’s kind of scary. Refer to a good Python reference text for all the gory details. Q: Can I inherit from my own custom classes? A: Of course, that’s the whole idea. You create a generic class that can then be “subclassed” to provide more specific, targeted functionality. Q: Can I put my class in a module file? A: Yes, that’s a really good idea, because it lets you share your class with many of your own programs and with other programmers. For instance, if you save your AthleteList class to a file called athletelist.py, you can import the into your code using this line of code: from athletelist import AthleteList then use the class as if it was defined in your current program. And, of course, if you create a really useful class, pop it into its own module and upload it to PyPI for the whole world to share. you are here 4 209 test drive Test Drive One last run of your program should confirm that it’s working to specification now. Give it a go in IDLE to confirm. 210 Chapter 6 Your entire program produces now the output the coach wants. Coach Kelly is impressed custom data objects That looks great! I can’t wait to show this to my young athletes and see their reaction... By basing your class on built-in functionality, you’ve leveraged the power of Python’s data structures while providing the custom solution your application needs. You’ve engineered a much more maintainable solution to Coach Kelly’s data processing needs. Good job! you are here 4 211 CHAPTER 6 python toolbox Your Python Toolbox You’ve got Chapter 6 under your belt and you’ve added some key Python techiques to your toolbox. Python Lingo •sat•“strD•s“hudoiiKecn“ccictVctietdaulayiutoiril”ocduenetneia-nai”dtrogrhaynt-yta”aahtarn(et-ywohav.tlhaoelalhiloocuedbkwhera-ussticdulwytaapaon-ittpupiahabnatrerskodttteaarynotousyfafc.tvtuahrlueee),. More Python Lingo ot•bh“jasetecltfal”wina-stysaanrmceeef.terhsodtoartghuemceunrtrent 212 Chapter 6 Create a empty dictionary using the dict() factory function or using {}. To access the value associated with the key Name in a dictionary called person, use the familiar square bracket notation: person['Name']. Like list and set, a Python’s dictionary dynamically grows as new data is added to the data structure. Populate a dictionary as you go: new_d = {} or new_d = dict() and then d['Name'] = 'Eric Idle' or do the same thing all in the one go: new_d = {'Name': 'Eric Idle'} The class keyword lets you define a class. Class methods (your code) are defined in much the same way as functions, that is, with the def keyword. Class attributes (your data) are just like variables that exist within object instances. The __init__() method can be defined within a class to initialize object instances. Every method defined in a class must provide self as its first argument. Every attribute in a class must be prefixed with self. in order to associate it data with its instance. Classes can be built from scratch or can inherit from Python’s built-in classes or from other custom classes. Classes can be put into a Python module and uploaded to PyPI. CHAPTER 6 7 web development Putting it all together This Web thing will never catch on...especially now that I have my trusty Underwood to keep me company... Sooner or later, you’ll want to share your app with lots of people. You have many options for doing this. Pop your code on PyPI, send out lots of emails, put your code on a CD or USB, or simply install your app manually on the computers of those people who need it. Sounds like a lot of work…not to mention boring. Also, what happens when you produce the next best version of your code? What happens then? How do you manage the update? Let’s face it: it’s such a pain that you’ll think up really creative excuses not to. Luckily, you don’t have to do any of this: just create a webapp instead. And, as this chapter demonstrates, using Python for web development is a breeze. this is a new chapter 213 caring is sharing It’s good to share The coach showed us your program running on his laptop...any chance me and my friends could also get access to our list of times? I’d love to show them to my dad... Coach young Kelly’s athletes You’re a victim of your own success. The new requests come flooding in right after Coach Kelly starts showing off your latest program. It appears that everyone wants access to the coach’s data! The thing is: what’s the “best way” to do this? 214 Chapter 7 You can put your program on the Web You’ll want to be able to share your functionality with lots of people... ...but you probably want only one version of your program “out there” that everyone accesses... web development ...and you need to make sure updates to your program are easy to apply. A “webapp” is what you want. If you develop your program as a Web-based application (or webapp, for short), your program is: • Available to everyone who can get to your website • In one place on your web server • Easy to upate as new functionality is needed But…how do webapps actually work? you are here 4 215 anatomy of a web request Webapps Up Close No matter what you do on the Web, it’s all about requests and responses. A web request is sent from a web browser to a web server as the result of some user interaction. On the web server, a web response (or reply) is formulated and sent back to the web browser. The entire process can be summarized in five steps. Step 1: Your user enters a web address, selects a hyperlink, or clicks a button in her chosen web browser. I just type the web address into my browser’s location bar and press Enter... Step 2: The web browser converts the user’s action into a web request and sends it to a server over the Internet. Hreerqeuecsotm. es a web The Internet Step 3: The web server receives the web request and has to decide what to do next. Web Server Hey, hello there...what’s this? A web request just for me? How nice... Deciding what to do next One of two things happen at this point. If the web request is for static content—such as an HTML file, image, or anything else stored on the web server’s hard disk—the web server locates the resource and returns it to the web browser as a web response. If the request is for dynamic content—that is, content that must be generated—the web server runs a program to produce the web response. 216 Chapter 7 Step 4: The web server processes the web request, creating a web response, which is sent back over the Internet to the waiting web browser. Web Server web development Here you go...a web response generated just for you. Enjoy! rHeesproencseo.mes a web The Internet The (potentially) many substeps of step 4 In practice, step 4 can involve multiple substeps, depending on what the web server has to do to produce the response. Obviously, if all the server has to do is locate static content and sent it back to the browser, the substeps aren’t too taxing, because it’s all just file I/O. However, when dynamic content must be generated, the sub steps involve the web server locating the program to execute, executing the located program, and then capturing the output from the program as the web response…which is then sent back to the waiting web browser. This dynamic content generation process has been standardized since the early days of the Web and is known as the Common Gateway Interface (CGI). Programs that conform to the standard are often referred to as CGI scripts. That’s exactly what I need. Thanks! Step 5: The web browser receives the web response and displays it on your user’s screen. you are here 4 217 webapp requirements What does your webapp need to do? Let’s take a moment to consider what you want your webapp to look like and how it should behave on your user’s web browser. You can then use this information to help you specify what your webapp needs to do. I guess I need a nice, friendly home page to kick things off, eh? Yeah...and I want to be able to get at my times easily... ...and once I’ve selected mine, I want them to look nice on my screen, so I can print them for my mom. 218 Chapter 7 web development There’s nothing like grabbing your pencil and a few blank paper napkins to quickly sketch a simple web design. You probably need three web pages: a “welcome” page, a “select an athlete” page, and a “display times” page. Go ahead and draw out a rough design on the napkins on this page, and don’t forget to draw any linkages between the pages (where it makes sense). you are here 4 219 back-of-the-napkin sketch § sTgdtrihasaeprplthahiyoctsmhaaeendwfpareaigbeenlaidnpklpy.to There’s nothing like grabbing your pencil and a few blank paper napkins to quickly sketch a simple web design. You probably need three web pages: a “welcome” page, a “select an athlete” page, and a “display times” page. You were to draw out a rough design on the napkins. You were to draw any linkages between the pages (where it made sense). Cacrbolauipacdtackitgohoeo’nsbntuathtttoathhtoeslenedhteaioestnmsp.dhleaeCytplsdihacaegaktnea’losit.snthlieanonkf“Sataetollhleglcetotth”tee’os WFaeotlrhclonemtoewe’,staotlliCmtoihnaagcthdyaKoteual’l.lyl’sfiWndebhsietree is my See you on the track! Timimg data for Sarah: Select an athlete from Sarah this list to work with: James Julie Mikey Select 2.18 2.21 2.22 Home Select another athlete. tpaTdhrtihesohepvloeliadtttyehhessi’ersrtldidhntaekwwtsesoabeblapaepacncakgtdgeeetsd.o 220 Chapter 7 Design your webapp with MVC Now that you have an idea of the pages your webapp needs to provide, your next question should be: what’s the best way to build this thing? Ask 10 web developers that question and you’ll get 10 different answers; the answer often depends on whom you ask. Despite this, the general consensus is that great webapps conform to the Model-View-Controller pattern, which helps you segment your webapp’s code into easily manageable functional chunks (or components): web development The Model The code to store (and sometimes process) your webapp’s data The View The code to format and display your webapp’s user interface(s) The Controller The code to glue your webapp together and provide its business logic By following the MVC pattern, you build your webapp in such as way as to enable your webapp to grow as new requirements dictate. You also open up the possibility of splitting the workload among a number of people, one for each component. Let’s build each of the MVC components for your webapp. you are here 4 221 build a model Model your data Your web server needs to store a single copy of your data, which in this case is Coach Kelly’s timing values (which start out in his text files). When your webapp starts, the data in the text files needs to be converted to AthleteList object instances, stored within a dictionary (indexed by athlete name), and then saved as a pickle file. Let’s put this functionality in a new function called put_to_store(). While your webapp runs, the data in the pickle needs to be available to your webapp as a dictionary. Let’s put this functionality in another new function called get_from_store(). When your webapp starts: james.txt julie.txt mikey.txt While your webapp runs: Twchoitaehcshai’nlslgldoeaftpiatchkele stored in a dictionary [‘Is this the right room for an argument?’, “No you haven’t!”, ‘When?’, “No you didn’t!”, “You didn’t!”, ‘You did not!’, ‘Ah! (taking out his wallet and paying) Just the five minutes.’, ‘You most certainly did not!’, “Oh no you didn’t!”, “Oh no you didn’t!”, “Oh look, this isn’t an argument!”, “No it isn’t!”, “It’s just contradiction!”, ‘It IS!’, ‘You just contradicted me!’, ‘You DID!’, ‘You did just then!’, ‘(exasperated) Oh, this is futile!!’, ‘Yes it is!’] sarah.txt The put_to_store() function [‘Is this the right room for an argument?’, “No you haven’t!”, ‘When?’, “No you didn’t!”, “You didn’t!”, ‘You did not!’, ‘Ah! (taking out his wallet and paying) Just the five minutes.’, ‘You most certainly did not!’, “Oh no you didn’t!”, “Oh no you didn’t!”, “Oh look, this isn’t an argument!”, “No it isn’t!”, “It’s just contradiction!”, ‘It IS!’, ‘You just contradicted me!’, ‘You DID!’, ‘You did just then!’, ‘(exasperated) Oh, this is futile!!’, ‘Yes it is!’] Tianlhlaeofdsiinctgthlieeondpaiacrtkyalestwoitrehd 222 Chapter 7 The get_from_store() function {'Sarah': AthleteList... , 'James': AthleteList... , 'Julie': AthleteList... , 'Mikey': AthleteList... } Afrdomicttiohnear“gyeotf_fArtohmle_tsetLoriest()s”rfetunucrtnieodn web development Here is the outline for a new module called athletemodel.py, which provides the functionality described on the previous page. Some of the code is already provided for you. Your job is to provide the rest of the code to the put_to_store() and get_from_store() functions. Don’t forget to protect any file I/O calls. import pickle from athletelist import AthleteList def get_coach_data(filename): # Not shown here as it has not changed since the last chapter. Yhttoehhureeenddetiaeoctdtapioocnfpoarudroleaymtienwtihteh files. def put_to_store(files_list): all_athletes = {} aTlirshtgiusomffeunnftci.lteinoanmiesscaaslleitdswsiotleh a tAondsadveont’theforget fpdoiicrcktlfieoiln(eaarnI/ydOtcoherearcokrs). return(all_athletes) def get_from_store(): all_athletes = {} Gftrreheottamtutrihtnteehdecdafitncoitlbeito,ehnseaory caller. BnaAeotdethdihcletttfiooeunLnraceitrsttyiuosrn.onsf return(all_athletes) you are here 4 223 model module Here is the outline for a new module called athletemodel.py, which provides the functionality described on the previous page. Some of the code is already provided for you. Your job was to provide the rest of the code to the put_to_store() and get_from_store() functions. You were not to forget to protect any file I/O calls. import pickle from athletelist import AthleteList def get_coach_data(filename): # Not shown here as it has not changed since the last chapter. def put_to_store(files_list): all_athletes = {} TioatnabdotkjdoeetchaettneahceidAnhsiatctfthtaihlilneoelcetn,eteat,eLru’aysrins.ntddaitta for each_file in files_list: ath = get_coach_data(each_file) all_athletes[ath.name] = ath try: itdunEhsisacteetcdaAhinoactnasehat.rltheyhlte.eetTLe“h’kisseetyn“a”ovmbainjleueectis”theis dSiacvteiotnhaeryenotfire tAothalepticekLleis.ts with open(‘athletes.pickle', ‘wb') as athf: pickle.dump(all_athletes, athf) except IOError as ioerr: print(‘File error (put_and_store): ' + str(ioerr)) Aapnrtdortyde/cotenx’tycoefuportrgfteiolte I/O code. return(all_athletes) def get_from_store(): all_athletes = {} Setnihmtepirldyeicprteiicaokdnlaetriyhn.eto What could be easier? try: with open(‘athletes.pickle', ‘rb') as athf: all_athletes = pickle.load(athf) except IOError as ioerr: print(‘File error (get_from_store): ' + str(ioerr)) return(all_athletes) Again…don’t ftorryg/eetxcyeoputr. 224 Chapter 7 web development Let’s test your code to ensure that it is working to specification. Type your code into an IDLE edit window and save your code into a folder that also includes the coach’s text files. Press F5 to import your code to the IDLE shell, and then use the dir() command to confirm that the import has been successful: >>> dir() ['AthleteList', '__builtins__', '__doc__', '__name__', '__package__', 'get_coach_data’, 'get_from_store', 'pickle', 'put_to_store'] Create a list of files to work with, and then call the put_to_store() function to take the data in the list of files and turn them into a dictionary stored in a pickle: >>> the_files = ['sarah.txt', 'james.txt', 'mikey.txt', 'julie.txt'] >>> data = put_to_store(the_files) AHtehrele’steaLll isotfs.the >>> data {'James Lee': ['2-34', '3:21', '2.34', '2.45', '3.01', '2:01', '2:01', '3:10', '2-22', '201', '2.01', '2:16'], 'Sarah Sweeney': ['2:58', '2.58', '2:39', '2-25', '2-55', '2:54', '2.18', '2:55', '2:55', '2:22', '2-21', '2.22'], 'Julie Jones': ['2.59', '2.11', '2:11', '2:23', '310', '2-23', '3:10', '3.21', '3-21', '3.01', '3.02', '2:59'], 'Mikey McManus': ['2:22', '3.01', '3:01', '3.02', '3:02', '3.02', '3:22', '2.49', '2:38', '2:40', '2.22', '2-31']} At this point, the athletes.pickle file should appear in the same folder as your code and text files. Recall that this file is a binary file, so trying to view it in IDLE or in your editor is not going to make much sense. To access the data, use the dictionary returned by the put_to_store() or get_from_store() functions. Use the existing data in the data dictionary to display each athlete’s name and date of birth: >>> for each_athlete in data: print(data[each_athlete].name + ' ' + data[each_athlete].dob) James Lee 2002-3-14 Sarah Sweeney 2002-6-17 Julie Jones 2002-8-17 Mikey McManus 2002-2-24 BatthytearAicbctuehtslseeistn,geyLotiuhsteca“dnnaatgmaee.t”aatndth“edorbes”t of Use the get_from_store() function to load the pickled data into another dictionary, then confirm that the results are as expected by repeating the code to display each athlete’s name and date of birth: >>> data_copy = get_from_store() >>> for each_athlete in data_copy: print(data_copy[each_athlete].name + ' ' + data_copy[each_athlete].dob) James Lee 2002-3-14 Sarah Sweeney 2002-6-17 Julie Jones 2002-8-17 Mikey McManus 2002-2-24 tTishhaaets depaxrtpoaedcuitnceedtdh, beeyxraepctututlry_ntetodh_edstsicaotmrieeo(n)aa.sry you are here 4 225 interface view View your interface With your model code written and working, it’s time to look at your view code, which creates your webapp’s user interface (UI). On the Web, UIs are created with HTML, the Web’s markup technology. If you are new to HTML, it is worth taking some time to become familiar with this critical web development technology. There’s lots of material on the Web and more than a few good books out there. bHTgr[ieeTaNhtcsMiooetstmdiLinesmg…ofternunrhpodoaetmntbyfottooMhorhsakpaintqergtuek.widhec§eatkw’tilrnyiet]gw.:he Hey, we hear you are getting into web development? We have a small module that we put together that might help you generate HTML. It’s a little rough, but it works. You’re more than welcome to use it for your projects, if you like. C(ModoestRoefv)iewthTeeHamead First YATE: Yet Another Template Engine Your friends over at the Head First Code Review Team heard you’re planning to write some code to generate HTML for your webapp’s UI. They’ve sent over some code that they swear will make your life easier. It’s a small library of HTML-generating helper functions called yate. The code was produced quickly and was originally designed to be “throw away,” so the team has provided it as is. It’s somewhat raw, but it should be OK. 226 Chapter 7 web development from string import Template def start_response(resp="text/html"): return('Content-type: ' + resp + '\n\n') def include_header(the_title): with open('templates/header.html') as headf: head_text = headf.read() header = Template(head_text) return(header.substitute(title=the_title)) Tdcoohdceuerm.e’eNsnotnoacttoimomnmu,econhrthsa,enlepyxthphleianrnega,!tjiuosnts,the def include_footer(the_links): with open('templates/footer.html') as footf: foot_text = footf.read() link_string = '' for key in the_links: link_string += '' + key + ' ' footer = Template(foot_text) return(footer.substitute(links=link_string)) def start_form(the_url, form_type="POST"): return('') def radio_button(rb_name, rb_value): return('' for item in items: u_string += '' + item + ' ' u_string += ' ' return(u_string) def header(header_text, header_level=2): return('' + header_text + ' ') def para(para_text): return('' + para_text + '
') you are here 4 229 template engine described Let’s get to know the yate code before proceeding with the rest of this chapter. For each chunk of code presented, you were to provide a written description of what you think it does: from string import Template Import the “Template” class from the standard library’s “string” fNoorte“rteshpe”.default module. This allows for simple string-substitution templates. def start_response(resp="text/html"): return('Content-type: ' + resp + '\n\n') This function takes a single (optional) string as its argument and uses it to create a CGI “Content-type:” line, with “text/html” as the default. Open the def include_header(the_title): template (which is file HTML), with open('templates/header.html') as headf: head_text = headf.read() rspuerbaodsvtiiditteudtine,“tainintdtleh”e. header = Template(head_text) return(header.substitute(title=the_title)) This function takes a single string as its argument and uses at the title for the start of a HTML page. The page itself is stored within a separate file in “templates/header.html”, and the title is substituted in as needed. “topaHfOfhrnTipeoldeeH_MvniTs(ldLiuwnMtebk)hdhs,sLitec”dri.htteiluciaenittsdmkiesopintliianantrintey,hedef include_footer(the_links): with open('templates/footer.html') foot_text = footf.read() link_string = '' for key in the_links: as footf: link_string += '' + key ' ' Turn the footer = Template(foot_text) dlinicktsioinntaoryaosftring, return(footer.substitute(links=link_string)) Similar to the “include_header” function, this one uses its single + fTHwoTehriiMcrsidnLl,ogobhskupastacackitelsf’istotralne into a string. string as wstuhhbiecsthtiteismutptelhadetnein.to its argument to create the end of a HTML page. The page itself is stored within a separate file in “templates/footer.html”, and the argument is used to dynamically create a set of HTML link tags. Based on how they are used, it looks like the argument needs to be a dictionary. 230 Chapter 7 web development This is “POST” toyrpi“cGalElyTe”i.ther def start_form(the_url, form_type="POST"): return('') This function returns the HTML markup, which terminates the form while allowing the caller to customize the text of the form’s “submit” button. A simple “for” loop does the trick. def radio_button(rb_name, rb_value): return('' for item in items: u_string += '' + item + ' ' u_string += ' ' return(u_string) Given a list of items, this function turns the list into a HTML unnumbered list. A simple “for” loop does all the work, adding a LI to the UL element with each iteration. def header(header_text, header_level=2): return('' + header_text + ' ') Create and return a HTML header tag (H1, H2, H2, and so on) with level 2 as the default.. The “header_text” argument is required. def para(para_text): return('' + para_text + '
') Enclose a paragraph of text (a string) in HTML paragraph tags. Almost not worth the effort, is it? you are here 4 231 no dumb questions Q: Where are the HTML templates used in the include_ header() and include_footer() functions? A: They are included with the yate module’s download. Go ahead and grab them from the Head First Python support website, and put them into a folder of your choice. Q: Why do I need yate at all? Why not include the HTML that I need right in the code and generate it with print() as needed? A: You could, but it’s not as flexible as the approach shown here. And (speaking from bitter experience) using a collection of print() statements to generate HTML works, but it turns your code into an unholy mess. Q: And you did this because you are using MVC? A: Partly, yes. The reason the MVC pattern is being followed is to ensure that the model code is separate from the view code, which are both separate from the controller code. No matter the size of the project, following MVC can make your life easier. Q: But surely MVC is overkill for something this small? A: We don’t think so, because you can bet that your webapp will grow, and when you need to add more features, the MVC “separation of duties” really shines. Let’s get to know the yate module even more. With the code downloaded and tucked away in an easy-tofind folder, load the module into IDLE and press F5 to take it for a spin. Let’s start by testing the start_ response() function. The CGI standard states that every web response must start with a header line that indictes the type of the data included in the request, which start_response() lets you control: >>> start_response() 'Content-type: text/html\n\n' >>> start_response("text/plain") The plus vdaerfiaatulitonCs GonI raestphoenmsee.header, 'Content-type: text/plain\n\n' >>> start_response("application/json") 'Content-type: application/json\n\n' The include_header() function generates the start of a web page and let’s you customizee its title: >>> include_header("Welcome to my home on the web!") '\n\nWelcome to my home on the web! \nWelcome to my home on the web!\n' TtbtohrhioesbweiasnlepclrrlluoowscoioieklnslssheaoadfvliebtaytnlloieynokdbuiirtftofwmiaceeubsClstbySyr,Sowbwfuostirelekrdi,n(omgNno’OwtrTietwhoobnrytrhtyyiho;siuisHt. ’TiYsnMomauLreba.wintNte)bo. te 232 Chapter 7 web development The include_footer() function produces HTML that terminates a web page, providing links (if provided as a dictionary). An empty dictionary switches off the inclusion of the linking HTML: >>> include_footer({'Home': '/index.html', 'Select': '/cgi-bin/select.py'}) ' \nHome Select \n
\n\n\n' >>> include_footer({}) '\n\n
\n 
用于工业通信和电机控制的单芯片驱动器
