Python Cheat Sheet

Support/Big Data/Document/My Docs/Python/Python_CheatSheet.MD

Installation/Config

PIP


-- list all installed packages
pip list

-- list only local installed packages in a virtual env
pip list --local

-- search a package
pip list|grep <packagename>

-- show the package location
pip show <packagename>

location of the globally installed packages

python -m site

>>> import site
>>> print(site.getsitepackages())'

or
>>> import sys
>>> sys.path

location of the locally installed packages

pythom -m site --user-site

Virtual Env (Python 3.3+)

venv is a package shipped with Python 3, which you can run using python3 -m venv.
It serves the same purpose as virtualenv, but only has a subset of its features.
virtualenv continues to be more popular than venv, especially since the former supports both Python 2 and 3.

[Difference between virtualenv and venv](https://virtualenv.pypa.io/en/latest/)

Install virtualenv

pip install virtualenv

upgrade virtualenv

pip install --upgrade virtualenv

using virtualenv to create a virtual env for a particular python version

virtualenv -p /usr/local/bin/python3.5 kevin-venv

Create a virtual env

python -m venv <venv-name>

python -m venv kevin-venv

include global python packages

python -m venv kevin-venv --system-site-packages

using --local to list only local installed packages

pip list --local

-- typically, you create a folder for your project first;
-- then create a virtual env within/insider of that project folder
>1. mkdir kevin_python_proj1
-- 2a and 2b is same; pick one you like
>2a. python -m venv kevin_python_proj1/.venv
>2b. cd mkdir kevin_python_proj1 && python -m venv .venv

Activate/Deactivate the vtirual env

activate

source <venv-name>/bin/activate
source kevin-venv/bin/activate

deactivate

deactivate

Delete a virtualenv

rm -rf <venv-name>
rm -rf kevin-venv

Store copy of python package versions

-- all packages
pip freeze > requirements.txt

-- only local installed packages
pip freeze --local > requirements.txt

Install all packages from requirements.txt

pip install -r requriments.txt

Tips and Tricks

Date, Time, DateTime, Timezone

[See this](https://docs.python.org/3/library/datetime.html#datetime.datetime.now)


from datetime import datetime, date, time, timezone, timedelta

current_time = datetime.now(timezone.utc)
print("The current time with UTC is: ", current_time.strftime('%Y-%m-%d %H:%M:%S %Z'))
#The current time with UTC is:  2022-02-15 21:50:52 UT ==> when I run this from laptop, it is 3:50:52 PM,
# this mean Chicago is UTC-6

yesterday = date.today() - timedelta(days=1)
dt = yesterday.strftime("%Y-%m-%d")
print("Yesterday is: ", dt)
# Yesterday is:  2022-02-14

# local time
print("The local time is: ", time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()));
# The local time is:  2022-02-15 15:50:03

switch value


a = 10
b = 30

# switch value, now a = 30, b = 10
a,b = b,a

Merging Dictionaries (Python 3.5+)


x = {'a': 1, 'b': 2}
y = {'c', 3, 'd': 4}

# merged = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
merged = {**x, **y}

# if have same key, then right one win
x = {'a': 1, 'b': 2}
y = {'a', 3, 'c': 4}

# merged = {'a': 3, 'b': 2, 'c': 4}
merged = {**x, **y}

Check if a module contain a class/method


# Using a list comprehension
import os
[name for name in dir(os) if 'uname' in name.lower()] # return ['uname', 'uname_result']

# After you find the class, you can use built-in help() to see the detail
help(os.uname)

# another example
import collections
[name for name in dir(collections) if 'tuple' in name.lower()] # return ['_tuplegetter', 'namedtuple']

help(collections.namedtuple)

1. Comprehensions

List Comprehensions

[expression for item in iterable] [expression for item in iterable if condition]

squares = [x * x for x in range(10)]

even_squares = [x * x for x in range(10) if x % 2 == 0]

# multiple loops
x_dimension = ['a', 'b']
y_dimension = [1, 2, 3]
matrix = [(x,y) for x in x_dimension for y in y_dimension]
#output: [('a', 1), ('a', 2), ('a', 3), ('b', 1), ('b', 2), ('b', 3)]

Tuple Comprehensions

there is no such thing. there is no tuple comprehension

Dictionary Comprehensions

{key_expression: value_expression for expression in iterable}

word = "dictionary"

# Method A:
letter_count1 = {letter: word.count(letter) for letter in word}

# Method B: better performance using set() to remove duplicated letter from word first
letter_count2 = {letter: word.count(letter) for letter in set(word)}

# with condition
old_salary = {'Kevin': 2000, 'Brandon': 400, 'Elena': 300}
new_salary = {k: v*1.5 if v < 500 else v for (k, v) in old_salary.items()}
#output: new_salary = {'Kevin': 2000, 'Brandon': 600.0, 'Elena': 450.0}

Set Comprehensions

{expression for item in iterable} {expression for item in iterable if condition}

random_nums = [3, 6, 9, 3, 9, 18, 20, 18]

great_tens = {num for num in random_nums if num > 10}

#output: {18, 20}

Generator Comprehensions

from pprint import pprint
some_nums = (num for num in range(5))
pprint(some_nums)
#output: <generator object <genexpr> at 0x00000264B87BCF90>

1. Functional Programming

Named Tuple

import collections
from pprint import pprint

Cruise = collections.namedtuple(
    "Cruise",
    [
        "year",
        "cruise_line",
        "ship",
        "days",
    ],
)

my_cruise = (
    Cruise(year=2016, cruise_line="RCL", ship="Independency of Seas", days=5),
    Cruise(year=2017, cruise_line="NCL", ship="Escape", days=7),
    Cruise(year=2018, cruise_line="NCL", ship="Bliss", days=7),
)

pprint(my_cruise)

# output
((Cruise(year=2016, cruise_line='RCL', ship='Independency of Seas', days=5),
 Cruise(year=2017, cruise_line='NCL', ship='Escape', days=7),
 Cruise(year=2018, cruise_line='NCL', ship='Bliss', days=7))

Filter()

# Method A: print only RCL cruise: using filter
rcl_cruise = tuple(
    filter(lambda cruise: cruise.cruise_line.upper() == "RCL", my_cruise)
)

pprint(rcl_cruise)
#output
(Cruise(year=2016, cruise_line='RCL', ship='Independency of Seas', days=5),)

# Method B: print only RCL cruise: using iterator
rcl_cruise2 = tuple(
    cruise for cruise in my_cruise if cruise.cruise_line.upper() == "RCL"
)

pprint(rcl_cruise2)

Map Function


## Method A: using map function
cruise_history = tuple(
    map(
        lambda cruise: {
            "cruise_line": cruise.cruise_line,
            "name": cruise.ship,
            "past_year": 2022 - cruise.year,
        },
        my_cruise,
    )
)

pprint(cruise_history)

#({'cruise_line': 'RCL', 'name': 'Independency of Seas', 'past_year': 6},
# {'cruise_line': 'NCL', 'name': 'Escape', 'past_year': 5},
# {'cruise_line': 'NCL', 'name': 'Bliss', 'past_year': 4})

# Method B: using generator comprehension and convert it to tuple
cruise_history2 = tuple(
    {
        "cruise_line": cruise.cruise_line,
        "name": cruise.ship,
        "past_year": 2022 - cruise.year,
    }
    for cruise in my_cruise
)

# Another great example: track for each cruise line, which year I took it

# Method A: hard-coded the cruise line dictionary object as initial value for reduce() function

def reducer(acc, val):
    acc[val.cruise_line].append(val.year)
    return acc

cruise_line_year_history = reduce(reducer, my_cruise, {"RCL": [], "NCL": [], "DISNEY": []})

pprint(cruise_line_year_history)
# {'DISNEY': [], 'NCL': [2017, 2018], 'RCL': [2016]}

# Method B: improve on top of Method A using collections.defaultdict
import collections

cruise_line_year_history2 = reduce(reducer, my_cruise, collections.defaultdict(list))

pprint(cruise_line_year_history2)
# defaultdict(<class 'list'>, {'RCL': [2016], 'NCL': [2017, 2018]})
# please notice result doesn't have 'DISNEY' in it

Reduce Function

# Method A: using reduce() function
from functools import reduce
total_cruise_days = reduce(lambda acc, cruise: acc + cruise.days, my_cruise, 0)

pprint(total_cruise_days)
#output: 19

# Method B: using sum() function instead of reduce()
total_cruise_days2 = sum(cruise.days for cruise in my_cruise)

Parallel Processing

Please watch below youtube by Dan Bader

- [multiprocessing](https://www.youtube.com/watch?v=aysceqdGFw8&list=PLP8GkvaIxJP1z5bu4NX_bFrEInBkAgTMr)

- [concurrent.futures](https://www.youtube.com/watch?v=0NNV8FDuck8&list=PLP8GkvaIxJP1z5bu4NX_bFrEInBkAgTMr)


# Method A
import multiprocessing

# Method B
import concurrent.futures

1. Class & Object

Dynamic property set and get: setattr()/getattr()

class Cruise():
    pass

cruise_2019 = Cruise()

first_key = 'cruise_line'
first_val = 'RCL Symphony'

second_key = 'cruise_days'
second_val = 7

setattr(cruise_2019, first_key, first_val)
setattr(cruise_2019, second_key, second_val)

print(cruise_2019.cruise_line, cruise_2019.cruise_days)
# RCL Symphony 7

cruise_2019_days = getattr(cruise_2019, second_key)
print(cruise_2019_days)
# 7

# convert dictionary to class object
cruise_2018_dict = {'cruise_line': 'NCL Bliss', 'cruise_days': 7}
cruise_2018 = Cruise()

for k, v in cruise_2018_dict.items():
    setattr(cruise_2018, k, v)

for k in cruise_2018_dict.keys():
    print(getattr(cruise_2018, k))

# NCL Bliss
# 7

1. List

Looping List with both index and item: enumerate

cruise_hist = ['NCL Bliss', 'NCL Escape', 'Disney Dreamer']

for idx, cruise in enumerate(cruise_hist):
    print(f"{idx}: {cruise}")

# Test
0: NCL Bliss
1: NCL Escape
2: Disney Dreamer

# using start=1
cruise_hist = ['NCL Bliss', 'NCL Escape', 'Disney Dreamer']

for idx, cruise in enumerate(cruise_hist, start=1):
    print(f"{idx}: {cruise}")

# Test
1: NCL Bliss
2: NCL Escape
3: Disney Dreamer

Loop multiple lists using zip()

cruise_lines = ['NCL Bliss', 'RCL Independency', 'Disney Dreamer']
cruise_days = [7, 5,  4]
cruise_years = [2018, 2016, 2015]

for cruise_line, cruise_day, cruise_year in zip(cruise_lines, cruise_days, cruise_years):
    print(
        f"I cruised with {cruise_line} in year {cruise_year} for {cruise_day} days")

# Test
I cruised with NCL Bliss in year 2018 for 7 days
I cruised with RCL Independency in year 2016 for 5 days
I cruised with Disney Dreamer in year 2015 for 4 days

# cruise_tuple is a tuple in below
for cruise_tuple in zip(cruise_lines, cruise_days, cruise_years):
    print(f"I cruised with {cruise_tuple} ")

# Test
I cruised with ('NCL Bliss', 7, 2018)
I cruised with ('RCL Independency', 5, 2016)
I cruised with ('Disney Dreamer', 4, 2015)

1. File Operation

Using Content Manager

with open('test.txt', 'rd') as f:
    file_contents = f.read()

words = file_contents.split(' ')
word_count = len(words)

1. Misc

Get Help

- In Python Interpreter, type 'help(dir)' - In Python Interpreter, type build-in 'dir(xyz)'

Make sure you import the module(xyz) you want to investigate before you run help(xyz) or dir(xyz)

Input secret information (e.g. password)

from getpass import getpass

username = input('Username: ')
password = getpass('Password: ')

Unpacking

cruise_2018 = ('NCL Bliss', 7)

# using _ as a variable is a convention in Python
# to indicate you are not use that variable
_, days_cruised_2018 = cruise_2018

print(days_cruised_2018) # 7

cruise_years = [2008, 2015, 2016, 2017, 2018, 2019]

first_cruise_yr, second_cruise_yr, *rest_cruise_yrs = cruise_years

print(f"my very 1st cruise is in year {first_cruise_yr}")
print(f"my 2nd cruise is in year {second_cruise_yr}")
print(f"my rest of cruise years are {rest_cruise_yrs}")

# Test
my very 1st cruise is in year 2008
my 2nd cruise is in year 2015
my rest of cruise years are [2016, 2017, 2018, 2019]

cruise_years = [2008, 2015, 2016, 2017, 2018, 2019]

first_cruise_yr, *rest_cruise_yrs, latest_cruise_yr = cruise_years

print(f"my very 1st cruise is in year {first_cruise_yr}")
print(f"my latest cruise year is ==>  {latest_cruise_yr}")
print(f"my rest of cruise years are {rest_cruise_yrs}")

# Test
my very 1st cruise is in year 2008
my latest cruise year is ==>  2019
my rest of cruise years are [2015, 2016, 2017, 2018]

Ternary Operator


cruise_2018 = 'NCL'
cruise_2019 = 'RCL'

favorite_flag = 'Y' if cruise_2019 == my_favorite_cruise else 'N'
print(f"my 2019 cruise is my favorite cruise? {favorite_flag}")

favorite_flag = 'Y' if cruise_2018 == my_favorite_cruise else 'N'
print(f"my 2018 cruise is my favorite cruise? {favorite_flag}")

# Test
my 2019 cruise is my favorite cruise? Y
my 2018 cruise is my favorite cruise? N

Large Number format

- Using underscore (\_) in declaration - Using ':," to set group seperator to ',' in f string

china_population = 1_300_000_000_000
usa_population = 330_000_000_000

print(f"China population = {china_population:,}")
print(f"USA {usa_population:,}")

# Test
China population = 1,300,000,000,000
USA 330,000,000,000

Comments