What is EMI?

Expose my ignorance. After my PhD, no one corrects or challenges my writing, coding, ML theory, the value of life, etc. I realised I need to take some notes on what I learnt every week.

EMI-10: Python. Generator

As written in EMI#1, I am still learning the use of generator and iterator. When I open .txt or .csv file, I normally use pandas for no reasons; however, if the file is too large to open, need to follow a different approach. Assume we have a text file (list_of_int.txt) below:

Example 10.1: Create a simple function

Example 10.2: Use generator comprehension

I asked the question to my coding sensei, Jacob Unna, and he taught me a few more things. When we call open, the operating system creates a filehandle to that file and doesn't close it until you explicitly tell it to. (So it's best to use with open() as f: to make sure it gets closed, otherwise, the app will just use more memory). As f is a generator already:

Example 10.3: Use with open() as f:

I knew the above solution (because it’s available anywhere if we google “how to open a text file in python”) but I have never thought much carefully before.

I wanted to use StopIteration in my code because I learnt. Fundamentally the below codes (Ex10.4 and Ex10.5) are the same:

Example 10.4:

Example 10.5:

EMI-11: Python. Defaultdict

defaultdict enables us to handle missing keys. A very nice article by Real Python summarised the use of defaultdict . I have used it in a simple implementation, such as below:

Then, one of my colleagues used in a different way with lambda . The below code finds the maximum value in the list, all_list

EMI-12: Python. Create a simple graph

I sometimes show and tell how to code quickly during my remote work. I wanted to plot a simple math function like y = 6x log_2(x) + 6x. The series of mistakes I did are below:

I then googled it and found a simpler solution.

Since np.linspace returns np.array object, so three codes give similar results if the number of elements in the array is larger (NOTE: np.linspace gives slightly different):

For plotting the graph, simply use matplotlib.pyplot

EMI-13: Python. int(x, base=10)

I guess anyone has used the simple int() build-in functions; but we are able to specify the base of input. When I was studying algorithms, the input value was the binary string format.

EMI-14: Python. zip()

I have used zip() countless time in order to obtain each element of two lists, such as:

However, I have never thought what the function actually does. In Python 3.8 documentation, zip() makes an iterator that aggregates elements from each of the iterables. I have explored different uses of zip() function at this time.


Example 14.1: zip(dict)

Example 14.2: compare to iter(dict)

Example 14.3: tuple(zip()), list(zip()), dict(zip())

Data scientist and machine learning engineer. PhD in Signal Processing for Neuroscience. https://www.linkedin.com/in/takashi-nakamura-004875a6/