Azure Storage Service Demo

This is intended to be run on the Python 2 kernel.
If you have not installed azure and azure-storage un-comment the pip below and run it.

In [ ]:
#!pip install --upgrade azure-storage==0.32.0
In [ ]:
import csv
import sys
import azure.storage
from azure.storage.table import TableService, Entity
from azure.storage.blob import BlockBlobService
from azure.storage.blob import PublicAccess
import time
print("hello at" + time.asctime(time.localtime(time.time())))

follow the instructions in the book to create a storage account. the one used here is "tutorial", but you will want to pick something else and replace that name everywhere below

In [ ]:
account = 'tutorial'
block_blob_service = BlockBlobService(account_name=account,
    account_key='your account key ending in==')
block_blob_service.create_container('datacont', 
                                    public_access=PublicAccess.Container)
In [ ]:
table_service = TableService(account_name=account, 
                             account_key='same key as above')
In [ ]:
if table_service.create_table('DataTable'):
    print("table created")
else:
    print("table already there")

Now you need a directory with a csv file. Our example will look like this

'experiment1', '1', '3/15/2002', 'exp1', 'this is the comment'

'experiment1', '2', '3/15/2002', 'exp2', 'this is the comment2'

'experiment2', '3', '3/16/2002', 'exp3', 'this is the comment3'

'experiment3', '4', '3/16/2002', 'exp4', 'this is the comment233'

now you need a directory datafiles that has four blobs of any type. call them exp1, exp2, exp3, exp4.
we have one ready for you if you are using the tutorial container. It is stored in /datadir. If you are running this somewhere else uncomment the lines in the next cell and run them
let's see what is there.

In [ ]:
#!wget https://SciengCloud.github.io/datadir.tar
#!tar -xf datadir.tar
In [ ]:
 with open('/datadir/experiments.csv', 'rb') as csvfile:
    csvf = csv.reader(csvfile, delimiter=',', quotechar='|')
    for item in csvf:
        print(item)

the code below will upload the data blobs into the container "datacont", then create the url and upload the metadata into the table "DataTable".

In [ ]:
 with open('/datadir/experiments.csv', 'rb') as csvfile:
    csvf = csv.reader(csvfile, delimiter=',', quotechar='|')
    for item in csvf:
        print item
        block_blob_service.create_blob_from_path(
            'datacont', item[3],
            "/datadir/"+item[3]
            )
        url = "https://"+account+".blob.core.windows.net/datacont/"+item[3]
        metadata_item = {'PartitionKey': item[0], 'RowKey': item[1], 
                 'description' : item[4], 'date' : item[2], 'url':url} 
        table_service.insert_entity('DataTable', metadata_item)
        

Next let's do a querry for experiment1 and project onto the urls. it should print the urls created above

In [ ]:
tasks = table_service.query_entities('DataTable', filter="PartitionKey eq 'experiment1'", select='url')
for task in tasks:
    print(task.url)

The output should look somethink like

https://escistore.blob.core.windows.net/datacont/exp1

https://escistore.blob.core.windows.net/datacont/exp2

try clicking on the links in your output

Next download the "azure storage explorer" and look at you table.

You will need to add the account with the key to the storage explore.