A Brief Introduction of Python

How Exactly is Python Used? 

Just as the internet-of-things is expanding to ordinary objects, Python language is being used in multiple fields thanks to its diverse functionality. It can run stats on returns in fintech, it can be scripted to run jobs in DevOps, it can collect, scrape, and clean data for academic research – it can even help social media marketing teams analyze their KPIs. 

Most articles online say it’s so popular due to its low barrier of entry. Many say it “comes with batteries” thanks to its comprehensive standard libraries. 

Features aside, many are likely learning Python simply because of its popularity. It’s been reported Python is one of the first programming languages taught in secondary schools. It’s one of the ‘easiest’ languages to learn and non-developers can pick it up quickly thanks to the standard libraries available. 

Enterprise-level teams are choosing Python because it can create scripts, if not full applications. Strict standards don’t need to be applied to Python, unlike C# or Java, so skill level and language fluency are more open. 

A user doesn’t have to be an expert to expertly use Python for their use-case. 

The versatility of the language explains why so many non-developers use it in their daily operations. It’s great for data processing, statistical analysis, finance, machine learning, deep learning, network automation, artificial intelligence, and so on. 

Python, however, is constantly evolving, so every cohort has different ways of using it and of course, each developer has their own way of writing. We’ve all turned to Stack Overflow, YouTube, or our even friends on Discord to work out a problem. It’s easy for someone’s hack to become our own standard. 

Python Graphic User Interfaces 

A.K.A. a Python GUI, is a great option for those not comfortable writing Python code.  

Consider a marketing manager trying to update a KPI dashboard, or a newly hired engineer adjusting to the company’s server configurations. Python needs to have accurate command-line arguments and any errors could lead to serious consequences. 

Teams can create guiderails by introducing a GUI into their system. Teams with a Python GUI can safely and confidently: 

• run Python scripts independently 
• provide a job-specific interface to enforce limited input 
• limit who can do what with a Python script in that given situation 
• give immediate visual feedback (aka lets you know when you’ve forgotten a required form, for example) 

Since they are meant to be intuitive and easy to use for anyone, it makes inaccessible technology accessible (especially for Linux). 

Teams can create original GUIs through their own system or tools like Tkinter. We recommend creating Python scripts in Otter’s GUI-based development tool. Otter can run the scripts, but also help with the learning curve of your company’s unique job processes. 

🐍 Jump to read about creating Python GUIs. 

Python Packages & Package Managers 

Developers and non-developers alike can rely on the open-source community to jump-start their projects thanks to the hundreds of thousands of Python packages available online. 

These packages have reusable code that can reduce a user’s build time and generally improve the quality of the final product (though it’s important to carefully vet a package before downloading and integrating it into your system). 

Developers can rely on these packages to cut down on production costs and avoid redundant work (why reinvent the wheel when it’s available and free on PyPi.org).  

Just ensure you have a Package Approval Process before letting a team have free reign on open-source sites. 

Packages can be installed with a single command using a package manager. 

Most Python users have a preferred manager. Some common ones are: 

PyPI: The Package Index 

Python’s own official third-party software repository. The Python Package Index (PyPI) is a repository of software that hosts an extensive collection of Python packages, development frameworks, tools, and libraries. 

Pip: The Standard Package Manager 

Pip is built into Python and can install packages from many different sources with PyPI.org as the primary and default package source used. 

Virtual Environments & Virtualenv 

In the Python world, a virtual environment is a folder containing packages and other dependencies that a Python project needs. The purpose of these environments is to keep projects separate and prevent dependency, version, and permission conflicts. 

Problems with Python Packages 

The downside to the huge availability of open-source Python packages can be boiled down to three main issues: 

#1: License Agreements 

Casually downloading a package from PyPI.org and integrating it into a project without checking it’s metadata could have serious consequences. 

Malicious packages or unacceptable licenses must be avoided while browsing open-source sites. Teams using PyPI.org or third-party packages should consider including Python packages into their licensing policy. 

Integrating Python packages into an organization’s existing third-party software policy allows developers to download from PyPI.org without constantly checking metadata because it’s already been done for them. 

🐍 Jump to read about License Detection in Python Packages. 

#2: Package Security Risks

A great way to prevent malicious packages (containing viruses, trackers, or malware) from making their way through to production is by setting up a Package Approval Workflow and Connector Filters.

It’s important to remember, however, that malicious packages are only one type of package security risk. Legitimate packages can also contain vulnerabilities that are discovered months or even years after publication, making ongoing vulnerability management just as important as package approval.

A package approval workflow is just like a code review but for open-source packages from PyPI. To use a package in a project, it’s reviewed by a trained “Approver.” Once reviewed and approved it’s made available to Python users, otherwise, it’s blocked from organization use.

This ensures that a trained set of eyes gets on every single PyPI package before it makes its way to Python coders or eventually production. The PyPI package is then put into a repository like ProGet so all Python users can easily access all the packages they need.

Connector filters can be introduced to any repository you have connecting to PyPI and can filter out pre-approved or even pre-denied packages. This helps streamline your package approval workflow and gets your developers the packages they need quickly.

Even with these controls in place, vulnerabilities can still be discovered after a package has already been approved and deployed. That’s why organizations should routinely scan packages and monitor for newly disclosed vulnerabilities throughout the software lifecycle.

🐍 Jump to read about Vulnerability Scanning in Python Packages. 

#3: Managing Python Dependencies 

Python packages often depend on other packages known as dependencies. These dependencies can have their own dependencies, resulting in a complicated dependency tree. 

If you’re building an application with Python and two packages require different versions of the same package, then Python will have a version conflict and your project may not build. 

Fortunately, there are two easy ways to deal with complex dependencies: 

• Use Requirements.txt for Repeatable Builds 
• Use Package Consumers to Track Dependencies 
• Automating your Python testing in CI/CD 

Utilizing requirements.txt files, a package approval workflow, and ProGet’s Package Consumer will help ensure predictable builds and keep unwanted packages out of them. 

In addition, incorporating automated Python testing in your CI/CD pipeline can enhance your development process. Automated testing catches potential issues immediately, and lightens the burden of manual testing. 

🐍 Jump to read about Python Dependencies. 

Creating Python Packages 

Many organizations choose to utilize third-party packages from PyPI, but many others create their own proprietary packages, never to be shared in the open-source community. 

Making a Python package is like making a zip file with metadata. There are multiple tools available to make a package like setuptools and then upload via twine, but ultimately it depends on the developer’s preferences. 

We recommend using the following four best practices when creating Python packages. Since Python users are so diverse, setting these standards can help make distribution of packages more efficient. 

1. Use One Repository & Wiki Per Library 
2. Keep Metadata Simple 
3. Use SemVer to help with Versioning and Dependency Tracking 
4. Use Wheels for Built Distribution 

🐍 Jump to read Authoring Best Practices.  

Integrating CI/CD into Your Python Development 

The two aren’t often discussed together, but CI/CD principles and best practices can 100% apply to Python development. 

Ever heard of a Python Pull and Pray? When you clone your application using git, install the dependencies with pip, and hope it all works out? This method causes a lot of problems: bloated deployment time; errors in the build; and no guarantee the app is the same server-to-server. 

Enter CI/CD for Python: create a stable base, clone and commit frequently, and test rapidly. Integrate Docker with BuildMaster for advantages like faster and simpler deployments and consistency across different environments. 

Setting up a Python CI/CD pipeline with Docker is easy and entirely customizable, allowing developers to optimize workflows and focus more on development tasks. 

🐍 Jump to read about Python CI/CD Practices. 

Ready to Start? 

We’ve curated this book into three distinct sections: Best Practices, Package Management, and Scripting Tools.  

Best Practices will help set standards within your team, who may or may not communicate their unique operations styles. This section will help set a baseline within the group, and set the team up for success way, way down the line.  

Package Management is all about, well, managing packages. We’ll discuss ways to protect your organization from lawsuits, vulnerabilities, and overall keeping things clean. 

Lastly, we’ll discuss some tools you can use to aid in Python scripting, like creating a custom GUI and using virtual environments.  

Ready to get started?

First, let’s look at Python package authoring and the practices that make packages easier to share and maintain.