Working With Numbers¶

Numeric calculations are usually done to initialize, change, or update the value of some property. The value of the calculation is assigned as the new value of the property. These properties are “variable properties” because their value can change. Instead of “variable properties” the term “variable” is often used as a shorthand. As we have seen, an example of the temperature conversion calculation in Python is

ftemp = 77
celsius = (ftemp - 32)/1.8

The result of this calculation, that is the value assigned to the variable celsius, is 25.0.

Calculations are largely straightforward but there are some dangers. A deceptively incorrect way to write the temperature conversion calculation is:

ftemp = 77
celsius = ftemp-32 / 1.8

In Python the value assigned to the variable celsius in this case is (approximately) 59.2. There is clearly a big difference between these two calculations.

The difference in the two calculations above is that in Python (as in most programming languages) there is a defined order in which the steps of a calculation are carried out. Each step in the calculation involves picking a mathematical operator and performing that operation on its operands. In each example above there is a choice between performing the subtraction step first or the division step first. In Python, each mathematical operator is giving a ranking, or “precedence” in the defined order. The list of numeric operators and their precedence is as follows:

where the parenthesis for grouping is the highest precedence while both addition and subtraction are the lowest precedence.

In the first example the parenthesis in the part (ftemp - 32) led to the subtraction being performed first. The steps in the first example are:

1. select the subtraction operator and compute the result of 77 - 32 which is 45
2. select the division operator and compute the result of 45/1.8 which is 25.0

In the second example the choice was made to perform the division step first because, without the parenthesis, the “precedence” of the division operator is higher than that of the the subtraction operation. The steps in the second example are:

1. select the division operator and compute the result of 32/1.8 which is (approximately) 17.7
2. select the addition operator and compute the result of 77 - 17.7 which is (approximately) 59.2

If you are not sure whether a calculation is written correctly you can always:

• test it. Use Python’s interactive ability to try the calculation and see if it is giving the expected results.
• add parenthesis. You can always be sure of the order of the steps in a calculation by adding parenthesis since this has the highest precedence.
• rewrite it. Break a single assignment down into several assignments where the steps in each assignment are clear. For example, the temperature conversion could be written as:

ftemp = 76
num = ftemp - 32
celsius = num/1.8

Working With Characters¶

Strings of characters are natural ways to represent texts (“To be or not to be”), persons (“John Doe”), locations (“Blacksburg, VA”), events (“Norman Invasion”), conditions (“hot”), stocks (“MS”), genes sequences (“AACGTTTAACC”), and many more. Many of the big data streams will have some form of character data included.

String of characters that look like numbers are not numbers. The character string “256” and the number 256 are very different (at least to a computer). The first is a character string and the second is a number. In general, operations that apply to one cannot be applied to the other. For example, it is not meaningful to write “256” - 50 because the character string “256” is not a number.

A Python string is a sequence of characters. Each character in the string has a position or index. The index of the first character is 0 (zero), the index of the second character is 1, and so on. If a string has n characters the index of the last character in the string is n-1. Blanks and spaces are characters even though they are not visible.

The code below shows some of the basics of working with strings. The built in len function returns the number of characters in the string. Individual characters can be accessed by their positions. For example, text[0] refers to the character in the first position. A slice is a consecutive sequence of characters. In this example text[0:10] refers to the 11 (eleven) consecutive characters beginning with the first character. If the first index in a slice is missing it is assumed to be the first character and if the second index in a slice is missing it is assumed to be the last character.

text = "To be or not to be. That is the question." 
print(text) 
length = len(text) 
print("has",length, "characters") 
print(text[0]) # individual characters print(text[1]) 
slice1 = text[0:10] # a string slice 
slice2 = text[10:20] 
print(slice1) 
print(slice2) 
slice3 = text[:15] # same as text[0:10] 
slice4 = text[35:] # from 35 to the end
print(slice3) 
print(slice4)

Strings can be broken apart and combined. The first part of the following example breaks a string in half. Note that integer division ("//") is used to yield an integer that can be used as an index. The second part of the example below combines two slices to form an ellipsis of the entire text. Notice that the plus sign (“+”) when applied to strings can be used to concatenate strings together.

text = "To be or not to be. That is the question." 
print(text) 
middle = len(text) // 2 # finding the middle using integer division 
start = text[0:middle] # first half of string 
rest = text[middle:] # second half of string 
print(start) 
print(rest) 
slice3 = text[:10] 
slice4 = text[35:] 
join = slice3 + "..." + slice4 # combining strings print(join)

Python provides a wide variety of ways to search and edit character strings. The operator in searches for a word contained in the string and yields True if the word is present and False otherwise. The count method finds how many times a word occurs in the string. The replace method produces a new string where the occurrences of one word is replaces with another word. Finally, the find method returns the index of where a specified word in the text begins.

text = "To be or not to be. That is the question." 
print(text) 
if "not" in text: # search text of an occurrence 
    print("contains the word 'not'") 
number = text.count("be") # search text for number of occurrences 
print("contains the word 'be'", number, "times") 
where = text.find("not") 
print("The word 'not' starts at", where) 
new_text = text.replace("be", "exist") # change word with another word 
print(new_text)

The details for these operations and a complete list of the built-in string methods can be found in the Python Documentation.

If Statements¶

We have seen in NetLogo and Blockly how programs can make decisions to guide the actions of the program. The decision statements in Python come in three “flavors” which are illustrated by the following examples that categorize a temperature according to this table:

According to this classification a temperature of 95 degrees would be “Hot”, 90 degrees would be “Warm”, 75 degrees would be “Cool” and 32 degrees would be “Cold”.

The indentation of code that was described earlier is a vital part of the Python syntax for decision statements. Recall that a standard practice in Python is to use four (4) spaces at each level of indentation.

In the first example we simply want to decide if a given temperature is classified as “Hot”. The flowchart form of the logic that is needed is shown in the following figure.

The Flowchart for Classifying “Hot” Temperatures

Here is the corresponding Python code:

temp = 80 # try changing the value of temp 
          # to get the output of "Hot" 
category = "Other" 
if temp > 90 : 
    category = "Hot" print(category)

There are three things to notice about the syntax of this code. They are:

• there is a colon ”:” at the end of the if statement.
• the statements that are to be executed when the condition is true are indented four more spaces than the if statement itself; in this case there is only one statement to be indented but, if there more, all of them must be similarly indented.
• the statement that follow the if statement are indented at the same level as the if statement itself; notice that the print and if statements are at the same level of indentation.

In the second example we want to decide if a given temperature is either “Hot” or “Frigid”. The flowchart form of the logic that is needed is shown in the following figure.

The Flowchart for Classifying “Hot” and “Frigid” Temperatures

Here is the code for this case:

temp = 72 # try changing the value of temp to get 
          # each of the three possible outputs 
category = "Other" 
if temp > 90 : # first decision 
    category = "Hot" 
else: 
    if temp <= 32:
        category = "Cold"

The important things to notice about the syntax of this example are:

• both if statements are ended by a colon (”:”)
• the keyword else begins the alternative to the first if and is also ended by a colon (”:”)
• the code for the else alternative is indented four more spaces than the else itself; it is indented at the same level as the code executed when the first decision is true; the indentation of the else code shows that it is to be executed only as an alternative to the first decision
• the code to be executed when the second decision is true is indented four spaces more than the if statement itself.

In the third example we want to decide if a given temperature is either “Hot”, “Warm”, “Cool” or “Other”. The flowchart form of the logic that is needed is shown in the following figure.

The Flowchart for Classifying “Hot”, “Warm”, “Cool” and “Other” Temperatures

Here is the code for this case:

temp = 72 # try changing the value of temp to get 
          # each of the four possible outputs 
if temp > 90 : 
    category = "Hot" 
elif temp > 76: 
    category = "Warm" 
elif temp > 50 : 
    category = "Cool" 
else: 
    category = "Other" print(category)

The important things to notice about the syntax of this example are:

• the elif keyword is a contraction of else if
• all if, elif and else lines are ended by a colon (”:”)
• the code to be executed for each decision is indented four more spaces that the decision itself

Conditions¶

The conditions used in if statements can involve a variety of individual tests and the individual tests can be combined to form complex conditions. Individual tests involve a comparison (or relational) operator. We have seen earlier individual test such as temp > 90 using the greater than operator. An individual test is either true or false. The comparison operators and their meaning are shown in the following table.

Logical (or Boolean) operators are used to combine individual tests into more complex tests. The logical (or Boolean) operators combine true and false values and result in a true or false value. The logical operators and their meaning are shown in the following table.

As an example, if we wanted to test for a temperature being either “Warm” or “Cold” we could use this code:

temp = 82 # try changing the value of temp to get 
          # each of the two possible outputs 
category = "Other" 
if (temp > 76): 
    category = "Warm" 
elif (temp < 32): 
    category = "Cold" 
print(category)

Alternatively, if we wanted to describes “Extreme” temperatures as those that are at either end of our temperature classification we could use this code:

temp = 92 # try changing the value of temp to get 
          # each of the two possible outputs 
category = "Other" 
if (temp > 90) or (temp <= 32):
    category = "Extreme"
print(category)

Finally, if we classified temperatures as “Seasonable” if it was not an extreme temperature we could use this code:

temp = 82 # try changing the value of temp to get 
          # each of the two possible outputs 
category = "Other" 
if not ((temp > 90) or (temp <= 32)):
    category = "Seasonal"
print(category)

Notice that an extra set of parenthesis is used to insure that the or operation is performed first.

Functions with One Parameter¶

The example shown below is the translated Blockly code that converts a stream of temperatures n Fahrenheit to a stream of temperatures in Celsius. Much of the code in this example should be easily understood: the for iteration, the calculation converting from one scale to the other, the list operations. The focus now is on the syntax of defining and using the function.

The general sense of the definition and use of the function in this example is:

Def*inition: I am the convert function. Give me a list, which I call *TheStream but you can call it whatever you like. I think this is a list of temperatures in Fahrenheit and I will give you back in return a list of corresponding Celsius temperatures. I call the list of Celsius temperatures TempStream but you can call it whatever you like.

Call: I have a list of Fahrenheit temperatures that I call Fstream. Please convert this to a list of Celsius temperatures. I will take the list that you return to me and call it Cstream.

import weather

def convert(TheStream): # function definition  
    TempStream = []  
    for temp in TheStream: 
        celsius = round((temp - 32) / 1.8) 
        TempStream.append(celsius) 
    return TempStream 

Fstream = weather.get_forecasts('Blacksburg, VA') 
print(Fstream) 
Cstream = convert(Fstream) # function call print(Cstream)
print(Cstream)

The definition of the function is begun in Python with the keyword def preceding the name of the function. In this case the name of the function is convert Following the name of the function is the “parameter” of the function in parenthesis. The “parameter” is the input to the function, in this case a list of temperature in Fahrenheit. The convert function has one parameter. A later example will show a function with more than one parameter. The function uses the list named TempStream to construct the list of Celsius temperatures. The return statement specifies the “output” of the function. When the function is actually executed the return statement signals the end of the execution of the function and returns the flow of control to the caller.

The code towards the bottom of the example shows how the function is used in the main program. The function call causes the list of Fahrenheit temperatures, called Fstream by the main program, to be converted to a list of Celsius temperatures, called Cstream in the main program. The function call simply involves using the name of the function and providing the expected input. The output returned by the function can be used wherever a value of the kind returned can appear.

Importing Modules¶

The main program in the example above uses functions that it does not define: the get_forecast function and the round function. It is typical that programs reuse predefined functions in this way. Why spend the effort reinventing what already exists? Collections of related functions are often grouped together into a module. Each module has a unique name.

To use one or more of the functions in a module that module must be import*ed. Importing means that the Python interpret will find a module with the given name and be prepared to use any of the function definitions that it finds in that module. The statement *import weather means that the Python interpreter will look for a module named weather. One of the functions that it will find defined in that module is the function get_forecasts.

To call a function in a module the caller must specify both the module name and the function name. The syntax requires a single period (”.”) between the module name and the function name. In the code above the statement weather.get_forecasts(...) is needed to name the function get_forecasts in the weather module.

The round function is an example of a “built in” function. Python has a set of predefined functions, such as round that are so common or basic that it provides for their use directly without the need for the module naming syntax.

The complete list of built-in and standard modules for Python can be found in the Python Standard Library . The Python developer community has an extensive collection of modules. We will use some of these libraries later.

Function with Multiple Parameters¶

A function may have as many parameters - as many inputs - as is needed for its purpose. The example below shows a function with two parameters. This function, named filter, has two parameters. The first parameter named aStream, is a list of temperatures. The second parameter, named limit is a temperature. The function returns a list of all of the temperatures in aStream that are at or above the limit temperature. That is, the function filters out all temperatures below the limit value.

def filter(aStream, limit): 
    SomeTemps = [] 
    for temp in aStream: 
        if temp >= limit: 
            SomeTemps.append(temp) 
    return SomeTemps

Notice in the example of the filter function that the two parameters are separated by a comma in parenthesis after the name of the function.

Structure and Operations¶

The structure of big data streams is, of course, more complex than the simple streams used as examples. The data stream of current temperatures for different cities may contain the information shown in the following table. This table contains four rows - each representing a city and its current temperature.

In Python each row in the table is called a key-value pair. The city is the key and the temperature is the value. The key is used in a table-like structure to find the row of interest. Because the key is used for this search the key must be unique in the table. For example, there can only be one row with the key “New York, NY”. The reason for this requirement is that if there were two rows with the same key the search would not know which one to use.

Python uses the term “dictionary” to describe a table like structure. Syntactically, the above table would be written as a Python dictionary as follows:

{ "Blacksburg, VA" : 77, "New York, NY" : 85, "San Jose, CA" : 75, "Miami, FL" : 88 }

where the dictionary is surrounded by open and closed curly brackets (“{” and “}”). Each row is separated from the next row by a comma (”,”). Each row is a key value pair where the key is separated from the value by a colon (”:”). Thus, “Blacksburg, VA” : 77 is a key-value pair where the string “Blacksburg, VA” is the key and the number 77 is the value.

A key differences between Python lists and Python dictionaries is that lists are ordered while dictionaries are not. In the table above, for example, the order of the rows does not make any difference because you can always find the temperature for any city in the dictionary regardless of where in the table the row for this city is placed. Finding something in a dictionary does not depend on the position of a row but the key value for that row.

The syntax of statements to find a row in a table and either access or change its value is shown in the following example. The example also shows how to add and delete key-value pairs.

temps = { "Blacksburg, VA" : 77, "New York, NY" : 85, 
           "San Jose, CA" : 75, "Miami, FL" : 88 } 

print(temps) 
bt = temps["Blacksburg, VA"] # access a value by its key 
print(bt) 
temps["New York, NY"] = 88 # change the value associated with a key 
print(temps) 
temps["Denver, CO"] = 68 # add a new key-value pair to the table 
print(temps) 
del temps["San Jose, CA"] # delete the key-value pair with the given key 
print(temps)

The syntax dictionary[entry] is used to denote a key-value pair where entry == key. If the key-value pair exists in the dictionary it can be used as the source for a value in an assignment statement or the target of an update operation to change its value. If the key-value pair does not exist and assignment can be used to create a new dictionary entry with the specified key and value. Finally, a key-value pair can be deleted from the dictionary using the del operation.

Add and delete elements to the above example to gain a better understanding of dictionaries.

Iteration and Dictionaries¶

Python provides an extension of the iteration operations to work with dictionaries. The following example shows a simple example of printing the contents of a dictionary. The method items() is needed to express the intent to iterate over the dictionaries key-value pairs.

temps = { "Blacksburg, VA" : 77, "New York, NY" : 85, 
          "San Jose, CA" : 75, "Miami, FL" : 88 } 

print(temps) 
for city, temp in temps.items(): 
    print( "city is:", city, " temperature is:", temp)

The additional syntax means that on each iteration through the dictionary the value of city represents the current element’s key and temp represents the current element’s value.

Another example of using iteration on dictionaries is the search code shown below. This search will print the temperature of all locations that have the state code “VA”. Notice that the dictionary in this example includes two cities in “VA” and the code prints a line for each city.

temps = { "Blacksburg, VA" : 77, "New York, NY" : 85, 
          "San Jose, CA" : 75, "Miami, FL" : 88, "Richmond, VA" : 79 } 

print(temps) 
state_code = "VA" 
for city, temp in temps.items(): 
    if(state_code in city): 
        print(city, "temperature is", temp)

Notice that they keyword in is used twice in this example. For character strings the “in” operator returns True if the left string appears somewhere in the right string and False otherwise. For example, “VA” in “Richmond, VA” is True and “VA” in New York, NY” is False.

Complex Dictionaries¶

It is often the case that the value part of a key-value pair is more than a single number. consider the table shown below. For each city in the table the nine day forecast for that city is given. The value in each row in this case is a list of numbers.

Python can easily represent this more complex arrangement of data. The dictionary corresponding to the above table is shown next.

temps = { "Blacksburg, VA" : [60, 71, 65, 66, 78, 75, 66, 55, 53] ,  
          "New York, NY" : [54, 50, 60, 63, 65, 61, 60, 59, 55] ,  
          "San Jose, CA" : [88, 89, 92, 93, 90, 88, 93, 87, 85] , 
          "Miami, FL" : [75, 80, 80, 81, 82, 79, 78, 50, 60] } 

for city, forecast in temps.items(): 
    print("Forecast for", city, "in three days is", forecast[2])

In this example the term forecast represents the list of temperatures forecast for city. The expression forecast[2] selects the third element of this list.