Today, professionals without formal training in software engineering are often driven to learn just enough Python syntax to hack together scripts and proof-of-concept code that seem to work and gets the job done. As soon as requirements grow in complexity, their tangled webs of glued-together code quickly reach dead ends and begin breaking in unexpected ways. Businesses that are scaling up realise the need for software engineers with deep understanding of fundamental computer science basics and their application to building production-quality software.
This book fills the knowledge gap, guiding novice coders to think like seasoned software engineers with a pragmatic, hands-on approach to building modern business software. It grounds programming fundamentals in real-world business objectives. You will gain mathematical clarity for taming complexity, while focusing on production-quality code.
Walking through essential data types and collections, the book bridges theory with practice. It introduces techniques for managing code complexity early on, instilling habits that prevent future pitfalls. You will learn reusable code patterns to construct reliable, maintainable systems.
With lucid explanations and thought-provoking exercises, the book develops computational thinking skills applicable across domains. It covers core concepts for efficient data manipulation, enhanced through Python's elegant syntax and teaches you how to write truly "Pythonic" code.
Finally, the book connects Python's versatile data analysis capabilities with the modern world of connected systems and networked data. You will learn the fundamentals of APIs and micro-services, readying them to participate in scalable distributed software architectures that are so widespread today.
Setting the Stage for Productive Learning
“Computer programming is an art, because it applies accumulated knowledge to the world, because it requires skill and ingenuity, and especially because it produces objects of beauty. A programmer who subconsciously views himself as an artist will enjoy what he does and will do it better.”
“It is rarely a mysterious technique that drives us to the top, but rather a profound mastery of what may well be a basic skill set.”
As with any other skill, it is not possible to master software engineering with Python in a few lessons. Learning typically takes years of dedication and practice, as you encounter different situations and gain confidence along the way in your ability to relate a new problem or challenge to something you already know.
What I hope this book will give you is a certain mindset and some principles which will accelerate your learning and give you a strong sense of “true north” from which to understand and evaluate new tools and technologies, as well as new code.
Some basic familiarity with programming concepts and Python itself is assumed, although most concepts are introduced in beginner terms. Even if you have more substantial experience with Python, the beginner perspective offered here will be beneficial, because it places (or "re-introduces") common programming concepts in the context of a broader framework of software engineering principles for structuring large-scale, production-quality code. So please, do not be irritated when you run into material you already know. Try to approach it with a beginner mindset and learn it afresh.
Through most of the chapters, you will be doing a fair amount of independent learning and research. The text will point the way and give you some context from the point of view of industry experience.
By the time you finish, regardless of how much you actually learn about Python itself, you will have a foundation and a good measure of clarity on how to go about expanding your knowledge and your skills in the directions which matter to you.
I have noticed that the great software engineers I have been lucky to work with are rarely made. The ones who stand out tend to be self-made. They are good at what they do because they are curious. They are driven from within to ask deeper questions, to explore, to try new things, to build, to create and to take things apart. They want to improve, to automate, to simplify, to generalise, to avoid repetition, to save time. You won't often see them memorising complex algorithms or the answers to common interview questions. But you will hear them talk about the "elegance" of a solution, or the "beauty" of a piece of code. These concepts are, of course, subjective, but once you have done a certain amount of solution-finding on your own, you will discover your own natural feel for what "beauty" and "elegance" mean in this context.
Good software engineers are sometimes more like artists than other types of engineers. While they do know their stuff, they also tend to be a little... let’s say “lazy” (in a loving way.) This turns out to be a good thing, because it drives them to automate things, to reuse and repurpose existing tools and solutions, so they don't have to do much manual work over and over again. After all, the essence of computing itself is to automate manual processes so we can do a lot more of them at scale in a cheaper and more reliable way.
Most importantly, even when they don't have the answer or the solution, they feel confident in their knowledge of where to look for it. This is why we will focus on building confidence in your ability to find the answers, more than on memorising or re-hashing existing solutions.
In school, perhaps you were encouraged to work independently and to not copy other people's work. I kind of want you to forget that for now. Ask questions. Search for things online. Talk to ChatGPT. Talk to people. Study their work. Copy it. Adapt it. Make it your own. Share yours. Get feedback. Find great solutions online and reuse those. But also do your best to understand how they did what they did and why it works. Remember also to ask yourself - could there be a better way of doing that?
There is rarely one perfect, ultimate solution. In practice, you will find that your problems (or your clients' problems) are sometimes unique in very specific ways. Once you understand all the constraints and requirements, you may be able to find various shortcuts or modifications to common solutions that fit better or are easier to implement. The key to doing that is understanding what actually matters in each case. How important is speed for this particular situation? How important is resource utilisation (e.g. memory usage)? How important is it for your solution to address all possible inputs, as opposed to a certain subset of inputs? What must be done now and what can be left for later? The answers to questions like these vary greatly and they end up guiding your approach more effectively than textbook patterns and algorithms.
A small remark on terminology. The word “Python” in this book can refer to more than one slightly different concept (none of which have anything to do with snakes in the natural world). Most typically, Python refers to the name of the programming language. This is what most people think of when they hear or speak about Python. However, the same word can also be used to refer to the Python interpreter, which is the piece of software which executes a Python program. That's what you are using when you run the python command on your computer. The interpreter, as the name suggests, reads the lines of code that make up your program (typically stored in a text file with a .py extension), interprets their meaning and builds an optimised in-memory representation of the Python program, which it then proceeds to execute.
Note that while there is an official standard Python interpreter (also known as "CPython”, because it is written in the C programming language) - which is almost certainly what you get on your computer when you run the python command - there exist multiple different, less popular implementations of a Python interpreter, such as PyPy (optimised for high-performance), Iron Python (meant to run on the Microsoft .NET framework), even Rust Python (a version of the python interpreter written in the modern language Rust) and others.
In the following chapters, we are going to dive straight into the essence of what computer programming is all about, exploring the idea of “abstraction” as well as the basic data types supported by the Python programming language. In the rest of the book, we will not be discussing anything other than the standard Python interpreter, but it may be helpful for you to have a cursory understanding of the alternatives.
When you are exploring the language, or just want to quickly try things out, it is convenient to use the Python interpreter directly, in an interactive mode. Just running python from the command line will open an interactive Python shell, in which you can type commands, one line at a time, and watch them get evaluated by the interpreter as soon as you hit Enter. There are even some fancy wrappers on top of the standard Python shell, which give you a more feature-rich environment on the command line - I personally quite like ipython and often use it as my default Python shell. ipython is not a different interpreter - it’s just a slightly more feature-rich shell on top of the standard Python interpreter. I do encourage you to download it and play around with it on your computer if you can.
However, once you are done exploring in the interactive shell and you have a clear idea of what you are doing, you will want to put your code in a Python text file, so it can be safely persisted on the file system. When naming your Python files, make sure the name reflects concisely but accurately the main purpose of your Python program or module. Try to only use lowercase letters in the name of the file, and no special characters - stick to letters and numbers and use underscores (_) instead of white space. This will ensure your files are as portable as they can be, so they can be directly copied and used on any operating system and all software tools (new or legacy) will be able to process them.
This book is meant to be read and worked through in a linear fashion. You will notice most chapters have questions and exercises at the end. This is to encourage you to do your own research and build up your confidence. The questions and exercises are as important, if not more important than the actual text of the lesson. Make sure to do your best to answer all the questions and complete all exercises to the best of your ability. If you get stuck and have someone you can ask for help, by all means ask. If you do not have a friendly Python guru at hand, do your best to find some help online (e.g. ChatGPT). If despite your best effort, you still don't feel you have an adequate answer or solution, do not get discouraged. Trust that the effort itself was worthwhile and carry on. After you work your way through the entire book, go back and attempt the hard questions again. Sometimes new ideas come to us after rest, or once we have gained certain new perspectives.
Do try to have fun with this! It is the best way to learn (or to do anything, really.) Another thing which will really help you understand new material is to try to explain or teach it to a friend or colleague. So if you can, help yourself by helping someone else. In recent years, "pair programming" has become a popular industry practice in companies that rely heavily on software engineering. I would like to suggest "pair learning" here. If you can, find a learning buddy and work through the book together, taking turns to ask and answer questions, work together through the exercises and challenge each other.
I will end with a quote from one of my favourite computer science educators:
“Your obligation [as a student] is that of active participation. You should not act as knowledge-absorbing sponges, but as whetstones on which we can all sharpen our wits”
Before you begin, here is a short checklist of things you should prepare:
- Ensure you have a recent version of Python (the code in this edition assumes Python 3.12 or newer) set up on your computer. You should know how to run the Python interpreter on the command line so you can interact with the Python shell. You should also know how to save Python code to a file and run it from the command line. Ensure you can do that with a simple script, like a one-line "Hello world" program.
- While not strictly necessary, I recommend you install a modern Python IDE (Integrated Development Environment) and get comfortable with it as your primary development environment. PyCharm by JetBrains is a great choice, currently widely used by professionals and novice programmers alike. It is a powerful tool worth getting used to, as it will help you a lot in the future, if you intend to do anything serious with Python. It also has some modern AI features. Once you download PyCharm, you should spend some time getting familiar with it - I recommend the official quick start guide. With this said, if you feel more comfortable with something like VSCode or a command-line editor such as Vim or Emacs, that is perfectly fine - these are also very powerful and flexible development environments.
- For the purposes of this book, please avoid using a system such as Jupyter or web-based “Python notebooks”. While those can be very convenient and quite popular collaboration tools in certain circles (e.g. in data science and other scientific disciplines), too often these types of interactive notebooks encourage a messy, unstructured end result, which is not at all conducive to the creation of maintainable, production-quality software.
- You should be able to use any operating system you feel comfortable with. I personally find Microsoft Windows to be somewhat less developer friendly than Unix-based operating systems such as Linux and MacOS. But the fact remains that you will be most productive with the OS you know best. The vast majority of the material and examples in the book (as well as the Python language itself) are not in any way OS-specific, although it is possible that occasionally you might need to make some small OS-specific adjustments to get your development environment to perform as expected. A mistake you should not be making, though, is to assume that if you only work on Windows, you will never need to learn or understand Unix concepts. The vast majority of software and Internet technologies today are built on top of traditional Unix concepts, and you will do well to familiarise yourself with the basics of Unix-based operating systems and a traditional command line shell such as Bash. No, that isn't strictly necessary for this book, so if you don't know much about Unix, please don't let that stop you - but it will make you a better programmer.
Table of Contents
- Introducing Fundamental Building Blocks - Data Types
- Working with Collections of Data
- The Dictionary Data Type and Conditional Logic
- Encapsulating Code into Reusable Functions
- Powerful Data Generation and Transformation Techniques
- The Essential Patterns of Data Manipulation
- On Code Quality and Managing Complexity
- Guiding Principles and Coding Best Practices
- Type Annotations and Reusability for Readable Code
- Dependency Management for Organised Projects
- Bundling Data with Behaviours using Classes and Objects
- Handling Exceptions Gracefully
- Working with Larger Datasets Stored in Files
- Gaining Confidence through Automated Testing
- Accessing Networked Data via APIs
For a PDF draft of the book (all of the above chapters) - contact jdimov@a115.co.uk