A very simple Tensorflow Logistic Regression example.

we will build a logistic regression model of some (fake) graduate school admissions decisions. The data consists csv file of a GRE exam score in the range 0 to 800, a grade point average in the range 0 to 4.0 and the rank of the student's undergrad institution in the range 4 to 1 (top). The Admission Decision in binary.

There are six columns in the data. Ignore column 0. the GRE, GPA and school rank are in columns 1 through 3. Column 4 is an admission decision that is only based on the rank of the school: only students with a rank 1 school get in and others not. Column 5 contains a more interesting (but still silly) admission model where you get in if your GRE score is 800 or your rank is 1. in the example below we use the former model. It is easy for the system to learn it perfectly.

The csv file is called dat.csv and it is here https://1drv.ms/u/s!AkRG9Zk_IOUag5IX1xzlv7rP9T5FFg

In [1]:
import tensorflow as tf
import numpy as np
import csv
In [2]:
sess = tf.InteractiveSession()
In [3]:
train_datas = []
train_labels = []
test_datas = []
test_labels = []
i = 0
scale = np.array([[0.01, 1.0, 1.0]])
with open('your path to/dat.csv', 'rb') as f:
    reader = csv.reader(f)
    for row in reader:
        if i < 300:
            train_datas.append([np.float32(x) for x in row[1:4]])
            #print np.float(row[0])
            train_labels.append([np.float32(row[4])])
        else:
            test_datas.append([np.float32(x) for x in row[1:4]])
            test_labels.append([np.float(row[4])])
        i +=1
In [4]:
test_data  = np.array(test_datas, dtype=np.float32)
test_label = np.array(test_labels, dtype=np.float32)
train_data  = np.array(train_datas, dtype=np.float32)
train_label = np.array(train_labels, dtype=np.float32)
In [5]:
test_data[0:20]
Out[5]:
array([[ 640.        ,    3.29999995,    2.        ],
       [ 660.        ,    3.5999999 ,    3.        ],
       [ 400.        ,    3.1500001 ,    2.        ],
       [ 680.        ,    3.98000002,    2.        ],
       [ 220.        ,    2.82999992,    3.        ],
       [ 580.        ,    3.46000004,    4.        ],
       [ 540.        ,    3.17000008,    1.        ],
       [ 580.        ,    3.50999999,    2.        ],
       [ 540.        ,    3.13000011,    2.        ],
       [ 440.        ,    2.98000002,    3.        ],
       [ 560.        ,    4.        ,    3.        ],
       [ 660.        ,    3.67000008,    2.        ],
       [ 660.        ,    3.76999998,    3.        ],
       [ 520.        ,    3.6500001 ,    4.        ],
       [ 540.        ,    3.46000004,    4.        ],
       [ 300.        ,    2.83999991,    2.        ],
       [ 340.        ,    3.        ,    2.        ],
       [ 780.        ,    3.63000011,    4.        ],
       [ 480.        ,    3.71000004,    4.        ],
       [ 540.        ,    3.27999997,    1.        ]], dtype=float32)
In [6]:
train_label[0:20]
Out[6]:
array([[ 0.],
       [ 0.],
       [ 1.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 1.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 1.],
       [ 1.],
       [ 0.],
       [ 1.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 1.]], dtype=float32)

first define the placeholders and variables

In [7]:
x = tf.placeholder(tf.float32, shape=(None,3)) 
y = tf.placeholder(tf.float32, shape =(None,1)) 

# Set model weights
W = tf.Variable(tf.zeros([3, 1]))
b = tf.Variable(tf.zeros([1]))
In [8]:
training_epochs = 400000
batch_size = 100
display_step = 1000
In [9]:
pred = tf.sigmoid(tf.matmul(x, W) + b) # Softmax
cost = tf.sqrt(tf.reduce_sum((y - pred)**2/batch_size))

opt = tf.train.AdamOptimizer()
optimizer = opt.minimize(cost)

# Initializing the variables old version is initialize_all_variable.  New version
# has global_variable_initializer.

#init = tf.global_variables_initializer()
init = tf.initialize_all_variables()
In [10]:
def get_batch2(batch_size, x, y):
    indices = np.random.choice(299, batch_size)
    return x[indices], y[indices]
In [14]:
sess.run(init)
# Training cycle
for epoch in range(training_epochs):
    avg_cost = 0.
    total_batch = int(len(train_data)/batch_size)
    # Loop over all batches
    for i in range(total_batch):
        batch_xs, batch_ys = get_batch2(batch_size, train_data, train_label)
        # Fit training using batch data
        _, c = sess.run([optimizer, cost], feed_dict={x:batch_xs,y:batch_ys})
        # Compute average loss
        avg_cost += c / total_batch
    # Display logs per epoch step
    if (epoch+1) % display_step == 0:
        print "Epoch:", '%04d' % (epoch+1), "cost=", str(avg_cost)

print "Optimization Finished!"
Epoch: 1000 cost= 0.122316965461
Epoch: 2000 cost= 0.0453300245106
Epoch: 3000 cost= 0.0223024163395
Epoch: 4000 cost= 0.0106212006882
Epoch: 5000 cost= 0.00716763352975
Epoch: 6000 cost= 0.0032860823907
Epoch: 7000 cost= 0.00211277613416
Epoch: 8000 cost= 0.00103540294804
Epoch: 9000 cost= 0.0010009273421
Epoch: 10000 cost= 0.00043358487892
Epoch: 11000 cost= 0.000273132210714
Epoch: 12000 cost= 0.000131385761779
Epoch: 13000 cost= 0.000147594412556
Epoch: 14000 cost= 8.70497853612e-05
Epoch: 15000 cost= 6.14167438471e-05
Epoch: 16000 cost= 4.52752279671e-05
Epoch: 17000 cost= 2.8381660195e-05
Epoch: 18000 cost= 1.55341675054e-05
Epoch: 19000 cost= 1.11633313281e-05
Epoch: 20000 cost= 4.60792912236e-06
Epoch: 21000 cost= 6.14725836385e-06
Epoch: 22000 cost= 4.10277780247e-06
Epoch: 23000 cost= 2.25179012432e-06
Epoch: 24000 cost= 1.33946480219e-06
Epoch: 25000 cost= 1.38382275736e-06
Epoch: 26000 cost= 5.88694848602e-07
Epoch: 27000 cost= 4.04469170689e-07
Epoch: 28000 cost= 3.81838839303e-07
Epoch: 29000 cost= 2.15980831797e-07
Epoch: 30000 cost= 2.29739438851e-07
Epoch: 31000 cost= 7.32606523002e-08
Epoch: 32000 cost= 7.81978350517e-08
Epoch: 33000 cost= 8.84199089057e-08
Epoch: 34000 cost= 6.10102098619e-08
Epoch: 35000 cost= 4.06389542462e-08
Epoch: 36000 cost= 2.8584245193e-08
Epoch: 37000 cost= 1.65094973248e-08
Epoch: 38000 cost= 2.87776908081e-08
Epoch: 39000 cost= 1.20515581514e-08
Epoch: 40000 cost= 1.39772810215e-08
Epoch: 41000 cost= 9.18894786928e-09
Epoch: 42000 cost= 8.88390081499e-09
Epoch: 43000 cost= 6.87570302915e-09
Epoch: 44000 cost= 9.23476699555e-09
Epoch: 45000 cost= 1.94402507425e-09
Epoch: 46000 cost= 6.34337748995e-09
Epoch: 47000 cost= 2.99907627621e-09
Epoch: 48000 cost= 5.28042193304e-10
Epoch: 49000 cost= 1.89618063473e-09
Epoch: 50000 cost= 6.51336373725e-10
Epoch: 51000 cost= 1.1945412326e-09
Epoch: 52000 cost= 1.5665533093e-09
Epoch: 53000 cost= 4.26993318481e-10
Epoch: 54000 cost= 1.61615453242e-09
Epoch: 55000 cost= 7.98362725929e-10
Epoch: 56000 cost= 2.8906340821e-09
Epoch: 57000 cost= 2.75020770879e-09
Epoch: 58000 cost= 4.38254998958e-10
Epoch: 59000 cost= 4.5128883408e-10
Epoch: 60000 cost= 7.8627946376e-10
Epoch: 61000 cost= 9.29251925408e-10
Epoch: 62000 cost= 3.97934674101e-10
Epoch: 63000 cost= 2.65806086308e-09
Epoch: 64000 cost= 2.52719324911e-09
Epoch: 65000 cost= 4.06026054445e-10
Epoch: 66000 cost= 4.12528758376e-10
Epoch: 67000 cost= 3.30549324024e-10
Epoch: 68000 cost= 3.47765782838e-10
Epoch: 69000 cost= 1.47295484143e-10
Epoch: 70000 cost= 2.16778331397e-10
Epoch: 71000 cost= 3.50019332962e-10
Epoch: 72000 cost= 1.66342456476e-10
Epoch: 73000 cost= 3.40016231837e-10
Epoch: 74000 cost= 5.31444244167e-10
Epoch: 75000 cost= 5.77119291245e-10
Epoch: 76000 cost= 1.8733346957e-10
Epoch: 77000 cost= 2.20593276978e-10
Epoch: 78000 cost= 3.31290700428e-10
Epoch: 79000 cost= 3.64823027343e-10
Epoch: 80000 cost= 3.35798952533e-10
Epoch: 81000 cost= 1.15360640485e-10
Epoch: 82000 cost= 1.61012950195e-10
Epoch: 83000 cost= 2.51071108348e-10
Epoch: 84000 cost= 1.91354737611e-10
Epoch: 85000 cost= 2.07550811954e-10
Epoch: 86000 cost= 1.25586371857e-10
Epoch: 87000 cost= 7.25016796599e-11
Epoch: 88000 cost= 3.02881156267e-10
Epoch: 89000 cost= 1.08787906461e-10
Epoch: 90000 cost= 7.06206863821e-11
Epoch: 91000 cost= 2.4414339661e-09
Epoch: 92000 cost= 9.18538980987e-11
Epoch: 93000 cost= 7.54220130794e-10
Epoch: 94000 cost= 3.31600424897e-10
Epoch: 95000 cost= 1.40610498939e-10
Epoch: 96000 cost= 2.5823253258e-10
Epoch: 97000 cost= 8.30305053878e-11
Epoch: 98000 cost= 8.33732791139e-11
Epoch: 99000 cost= 3.160616803e-11
Epoch: 100000 cost= 6.08583655692e-11
Epoch: 101000 cost= 3.86398607555e-10
Epoch: 102000 cost= 1.97125425705e-10
Epoch: 103000 cost= 9.48131251288e-10
Epoch: 104000 cost= 1.48454845639e-10
Epoch: 105000 cost= 3.02259365106e-11
Epoch: 106000 cost= 2.40462353536e-09
Epoch: 107000 cost= 2.88124332726e-10
Epoch: 108000 cost= 2.50083992404e-10
Epoch: 109000 cost= 8.99044262648e-11
Epoch: 110000 cost= 6.6031766166e-11
Epoch: 111000 cost= 4.08098638216e-11
Epoch: 112000 cost= 2.4110388807e-09
Epoch: 113000 cost= 2.94395210987e-10
Epoch: 114000 cost= 5.28096379127e-11
Epoch: 115000 cost= 2.14434969781e-11
Epoch: 116000 cost= 4.22186462873e-11
Epoch: 117000 cost= 3.52107343282e-11
Epoch: 118000 cost= 1.60918208446e-11
Epoch: 119000 cost= 1.79143422319e-10
Epoch: 120000 cost= 8.46240237606e-11
Epoch: 121000 cost= 1.7802278228e-11
Epoch: 122000 cost= 2.37734658348e-11
Epoch: 123000 cost= 2.31932806294e-11
Epoch: 124000 cost= 1.72319030572e-11
Epoch: 125000 cost= 2.95317330312e-10
Epoch: 126000 cost= 2.41031786995e-09
Epoch: 127000 cost= 1.73923783708e-10
Epoch: 128000 cost= 4.7645276266e-11
Epoch: 129000 cost= 2.15888643768e-11
Epoch: 130000 cost= 2.98203156612e-10
Epoch: 131000 cost= 9.879523033e-11
Epoch: 132000 cost= 2.26924033384e-11
Epoch: 133000 cost= 1.76252987899e-11
Epoch: 134000 cost= 1.97019251608e-11
Epoch: 135000 cost= 1.63373242018e-11
Epoch: 136000 cost= 3.38600855358e-09
Epoch: 137000 cost= 8.80321635033e-11
Epoch: 138000 cost= 3.25889419617e-11
Epoch: 139000 cost= 1.55772155591e-11
Epoch: 140000 cost= 1.42767853789e-10
Epoch: 141000 cost= 1.16555691487e-10
Epoch: 142000 cost= 1.42716494178e-10
Epoch: 143000 cost= 1.41315918169e-11
Epoch: 144000 cost= 3.07455877424e-10
Epoch: 145000 cost= 3.07468995403e-10
Epoch: 146000 cost= 9.29911338904e-12
Epoch: 147000 cost= 3.42092088493e-09
Epoch: 148000 cost= 1.21008479279e-11
Epoch: 149000 cost= 2.23882626327e-10
Epoch: 150000 cost= 3.62730724623e-10
Epoch: 151000 cost= 1.60480849096e-11
Epoch: 152000 cost= 7.0191736104e-12
Epoch: 153000 cost= 1.03193087755e-11
Epoch: 154000 cost= 9.62860338335e-12
Epoch: 155000 cost= 9.1627420061e-12
Epoch: 156000 cost= 1.08890775737e-11
Epoch: 157000 cost= 1.14331606682e-11
Epoch: 158000 cost= 7.96990130142e-12
Epoch: 159000 cost= 1.89369155773e-11
Epoch: 160000 cost= 2.36491788652e-10
Epoch: 161000 cost= 8.30489600701e-12
Epoch: 162000 cost= 9.86834962705e-12
Epoch: 163000 cost= 7.91284008803e-12
Epoch: 164000 cost= 3.92040819946e-11
Epoch: 165000 cost= 8.50568357066e-12
Epoch: 166000 cost= 7.86828692478e-12
Epoch: 167000 cost= 1.09551374916e-11
Epoch: 168000 cost= 6.93246430313e-12
Epoch: 169000 cost= 1.37308046877e-11
Epoch: 170000 cost= 8.91637831277e-12
Epoch: 171000 cost= 1.38325661689e-11
Epoch: 172000 cost= 7.09427959086e-12
Epoch: 173000 cost= 7.02491302976e-12
Epoch: 174000 cost= 4.8248658957e-10
Epoch: 175000 cost= 1.24742243271e-10
Epoch: 176000 cost= 3.81960109375e-11
Epoch: 177000 cost= 7.30345280781e-12
Epoch: 178000 cost= 2.05650677043e-11
Epoch: 179000 cost= 1.36832670194e-10
Epoch: 180000 cost= 5.66974115226e-12
Epoch: 181000 cost= 6.70359401669e-12
Epoch: 182000 cost= 8.34544718054e-11
Epoch: 183000 cost= 5.8632185912e-11
Epoch: 184000 cost= 1.52035946333e-11
Epoch: 185000 cost= 7.97635299823e-12
Epoch: 186000 cost= 3.53504073236e-11
Epoch: 187000 cost= 6.57172172583e-12
Epoch: 188000 cost= 7.03508631558e-12
Epoch: 189000 cost= 8.25382479377e-12
Epoch: 190000 cost= 5.089554906e-12
Epoch: 191000 cost= 6.93948986816e-12
Epoch: 192000 cost= 6.49243354391e-12
Epoch: 193000 cost= 1.41514763449e-11
Epoch: 194000 cost= 4.32684246071e-11
Epoch: 195000 cost= 8.60223801913e-12
Epoch: 196000 cost= 2.31290474273e-10
Epoch: 197000 cost= 5.18298069394e-12
Epoch: 198000 cost= 2.41364238041e-09
Epoch: 199000 cost= 2.21837780784e-11
Epoch: 200000 cost= 3.42594221221e-12
Epoch: 201000 cost= 1.73456327426e-11
Epoch: 202000 cost= 6.56342844657e-12
Epoch: 203000 cost= 4.86039125613e-12
Epoch: 204000 cost= 5.4290663111e-12
Epoch: 205000 cost= 1.96322494372e-10
Epoch: 206000 cost= 5.15717954452e-12
Epoch: 207000 cost= 8.34348354295e-12
Epoch: 208000 cost= 2.39624513591e-09
Epoch: 209000 cost= 4.48158411423e-12
Epoch: 210000 cost= 4.64099054326e-12
Epoch: 211000 cost= 4.25229126216e-11
Epoch: 212000 cost= 7.93938638791e-12
Epoch: 213000 cost= 1.41355923494e-10
Epoch: 214000 cost= 3.53960402655e-12
Epoch: 215000 cost= 9.80445776649e-12
Epoch: 216000 cost= 3.49412421739e-12
Epoch: 217000 cost= 2.40582304183e-09
Epoch: 218000 cost= 4.90450471034e-12
Epoch: 219000 cost= 3.86565315348e-12
Epoch: 220000 cost= 3.4912332747e-10
Epoch: 221000 cost= 6.28989664173e-12
Epoch: 222000 cost= 4.10026470242e-12
Epoch: 223000 cost= 2.9878081953e-11
Epoch: 224000 cost= 8.5767751408e-12
Epoch: 225000 cost= 4.06925868869e-12
Epoch: 226000 cost= 7.0098316908e-12
Epoch: 227000 cost= 4.40571745752e-12
Epoch: 228000 cost= 7.32331789655e-10
Epoch: 229000 cost= 3.36930761069e-12
Epoch: 230000 cost= 4.45795687691e-12
Epoch: 231000 cost= 3.93548605163e-12
Epoch: 232000 cost= 4.13768212401e-12
Epoch: 233000 cost= 1.856903728e-11
Epoch: 234000 cost= 3.20429798147e-12
Epoch: 235000 cost= 2.86679538615e-12
Epoch: 236000 cost= 5.98512905903e-12
Epoch: 237000 cost= 4.32772697886e-12
Epoch: 238000 cost= 3.83216206164e-12
Epoch: 239000 cost= 3.99749521679e-12
Epoch: 240000 cost= 8.71623086207e-12
Epoch: 241000 cost= 1.1905953827e-11
Epoch: 242000 cost= 3.89040115227e-12
Epoch: 243000 cost= 5.11123495958e-12
Epoch: 244000 cost= 5.88311760419e-12
Epoch: 245000 cost= 4.01351716618e-12
Epoch: 246000 cost= 4.92805778823e-12
Epoch: 247000 cost= 2.79269806065e-12
Epoch: 248000 cost= 2.03340653343e-11
Epoch: 249000 cost= 7.7896545117e-12
Epoch: 250000 cost= 3.39376836109e-09
Epoch: 251000 cost= 3.71519343068e-12
Epoch: 252000 cost= 2.51736236023e-12
Epoch: 253000 cost= 2.92845348665e-12
Epoch: 254000 cost= 2.06076683495e-12
Epoch: 255000 cost= 3.41454087227e-10
Epoch: 256000 cost= 6.1314653875e-12
Epoch: 257000 cost= 2.8170209004e-12
Epoch: 258000 cost= 2.69305628145e-12
Epoch: 259000 cost= 3.60264314041e-12
Epoch: 260000 cost= 3.73052951377e-12
Epoch: 261000 cost= 3.78640530388e-12
Epoch: 262000 cost= 2.80702895823e-12
Epoch: 263000 cost= 7.09909058622e-12
Epoch: 264000 cost= 3.540412798e-12
Epoch: 265000 cost= 1.11175237419e-11
Epoch: 266000 cost= 2.54337518928e-12
Epoch: 267000 cost= 4.16367201134e-12
Epoch: 268000 cost= 6.46051158751e-11
Epoch: 269000 cost= 3.42896386198e-12
Epoch: 270000 cost= 2.74427494618e-12
Epoch: 271000 cost= 8.54508668685e-12
Epoch: 272000 cost= 2.43406885359e-12
Epoch: 273000 cost= 5.50300492674e-11
Epoch: 274000 cost= 2.15107844731e-12
Epoch: 275000 cost= 3.2431662824e-12
Epoch: 276000 cost= 3.13241923719e-12
Epoch: 277000 cost= 3.42459979645e-12
Epoch: 278000 cost= 2.93905151952e-12
Epoch: 279000 cost= 8.62813301708e-12
Epoch: 280000 cost= 2.84016185136e-12
Epoch: 281000 cost= 1.35707466098e-10
Epoch: 282000 cost= 2.50335368754e-12
Epoch: 283000 cost= 2.39593730155e-09
Epoch: 284000 cost= 3.78944046281e-12
Epoch: 285000 cost= 3.01637154924e-12
Epoch: 286000 cost= 4.75307449649e-12
Epoch: 287000 cost= 3.02805393607e-12
Epoch: 288000 cost= 3.17096203895e-12
Epoch: 289000 cost= 4.7761467524e-12
Epoch: 290000 cost= 6.1802018364e-12
Epoch: 291000 cost= 3.78010838777e-12
Epoch: 292000 cost= 7.62586677833e-12
Epoch: 293000 cost= 2.29550147337e-12
Epoch: 294000 cost= 2.19209058457e-12
Epoch: 295000 cost= 2.86150248462e-12
Epoch: 296000 cost= 2.9989248064e-12
Epoch: 297000 cost= 1.35131176249e-09
Epoch: 298000 cost= 1.88376859684e-12
Epoch: 299000 cost= 2.04185045607e-12
Epoch: 300000 cost= 3.44196181973e-12
Epoch: 301000 cost= 2.65800532626e-12
Epoch: 302000 cost= 3.27015120712e-12
Epoch: 303000 cost= 1.91394810157e-12
Epoch: 304000 cost= 2.43867823071e-12
Epoch: 305000 cost= 6.27282982502e-12
Epoch: 306000 cost= 2.26029936955e-12
Epoch: 307000 cost= 2.31234610452e-12
Epoch: 308000 cost= 5.12209693063e-12
Epoch: 309000 cost= 2.2028755339e-12
Epoch: 310000 cost= 2.07901569224e-12
Epoch: 311000 cost= 2.15347997681e-12
Epoch: 312000 cost= 3.74108430866e-12
Epoch: 313000 cost= 2.73432444221e-12
Epoch: 314000 cost= 3.03151627065e-12
Epoch: 315000 cost= 1.94097554869e-12
Epoch: 316000 cost= 3.53397415498e-12
Epoch: 317000 cost= 2.16181499785e-12
Epoch: 318000 cost= 3.17850541894e-12
Epoch: 319000 cost= 2.70348040829e-12
Epoch: 320000 cost= 2.57664576741e-11
Epoch: 321000 cost= 2.15451094465e-12
Epoch: 322000 cost= 5.0700588259e-12
Epoch: 323000 cost= 1.5741229392e-12
Epoch: 324000 cost= 2.74990932803e-12
Epoch: 325000 cost= 7.31330742493e-12
Epoch: 326000 cost= 1.33724368384e-12
Epoch: 327000 cost= 1.74383109949e-12
Epoch: 328000 cost= 2.99310281421e-12
Epoch: 329000 cost= 3.73211054075e-12
Epoch: 330000 cost= 1.25843167484e-11
Epoch: 331000 cost= 2.76375328873e-12
Epoch: 332000 cost= 3.13655776699e-12
Epoch: 333000 cost= 1.6911756392e-12
Epoch: 334000 cost= 1.18459013215e-12
Epoch: 335000 cost= 2.46506274594e-12
Epoch: 336000 cost= 1.70862781389e-12
Epoch: 337000 cost= 2.93413730812e-12
Epoch: 338000 cost= 2.54218833484e-12
Epoch: 339000 cost= 2.04385874544e-12
Epoch: 340000 cost= 2.12691119057e-12
Epoch: 341000 cost= 1.77426317995e-12
Epoch: 342000 cost= 6.94020856409e-12
Epoch: 343000 cost= 1.54444602723e-12
Epoch: 344000 cost= 2.33288158756e-12
Epoch: 345000 cost= 4.50499547389e-10
Epoch: 346000 cost= 3.48901419908e-12
Epoch: 347000 cost= 3.22709808875e-10
Epoch: 348000 cost= 2.21275510935e-12
Epoch: 349000 cost= 1.74617419049e-12
Epoch: 350000 cost= 2.10434256825e-11
Epoch: 351000 cost= 3.77203966846e-12
Epoch: 352000 cost= 2.29987279221e-12
Epoch: 353000 cost= 1.69556437033e-12
Epoch: 354000 cost= 1.33359710774e-10
Epoch: 355000 cost= 2.57870248592e-12
Epoch: 356000 cost= 1.41810460944e-12
Epoch: 357000 cost= 1.9538651079e-12
Epoch: 358000 cost= 1.69699719771e-12
Epoch: 359000 cost= 1.13118510869e-12
Epoch: 360000 cost= 1.70162070198e-12
Epoch: 361000 cost= 2.72100115876e-12
Epoch: 362000 cost= 1.73955051752e-12
Epoch: 363000 cost= 2.0404165987e-12
Epoch: 364000 cost= 2.34752555936e-12
Epoch: 365000 cost= 2.77385566773e-12
Epoch: 366000 cost= 3.24670529177e-12
Epoch: 367000 cost= 3.37563210891e-12
Epoch: 368000 cost= 2.41833671144e-12
Epoch: 369000 cost= 1.88041663404e-12
Epoch: 370000 cost= 1.20623310602e-12
Epoch: 371000 cost= 1.79514907713e-12
Epoch: 372000 cost= 7.54782156914e-12
Epoch: 373000 cost= 3.80041800807e-10
Epoch: 374000 cost= 3.62610935202e-12
Epoch: 375000 cost= 7.53846564165e-12
Epoch: 376000 cost= 1.83012984108e-12
Epoch: 377000 cost= 1.87283905832e-12
Epoch: 378000 cost= 1.63762540854e-12
Epoch: 379000 cost= 2.46305736223e-12
Epoch: 380000 cost= 8.26208110261e-10
Epoch: 381000 cost= 2.48751726682e-12
Epoch: 382000 cost= 1.52378777998e-12
Epoch: 383000 cost= 1.24206609624e-12
Epoch: 384000 cost= 1.77230493736e-12
Epoch: 385000 cost= 6.00400172288e-12
Epoch: 386000 cost= 1.57113935577e-12
Epoch: 387000 cost= 1.43682021576e-12
Epoch: 388000 cost= 7.7820164375e-12
Epoch: 389000 cost= 1.72555270853e-12
Epoch: 390000 cost= 1.73287459685e-12
Epoch: 391000 cost= 1.27501656802e-11
Epoch: 392000 cost= 3.67395176781e-12
Epoch: 393000 cost= 9.21749402499e-11
Epoch: 394000 cost= 2.06389517022e-12
Epoch: 395000 cost= 1.38738376256e-12
Epoch: 396000 cost= 1.19345994675e-12
Epoch: 397000 cost= 1.33574532728e-12
Epoch: 398000 cost= 2.86569652556e-12
Epoch: 399000 cost= 5.18513626133e-12
Epoch: 400000 cost= 1.47335720013e-12
Optimization Finished!

define a function in python to compute sigmoid of an array.

In [ ]:
def sigmoida(z):
    res = np.zeros(len(z))
    for i in range(len(z)):
        y = 1.0/(1.0+np.exp(-z[i]))
        if y > 0.5:
            res[i] = 1
        else:
            res[i] = 0
    return res
In [23]:
p = sigmoida(pred.eval({x:test_data}))
q = y.eval({y:test_label})

how accurate is this. In other words whaat fraction of the cases have p = q.

In [24]:
def accurate(p,q):
    total = 0.0
    numones = 0.
    numzeros = 0.
    for i in range(len(p)):
        if p[i] == 0:
            numzeros += 1
        else:
            numones += 1
        if p[i] == q[i]:
            total +=1
    print "accuracy = "+str(total/len(p))
    print 'ratio of 1s ='+str(numones/len(p))
    print 'ratio of 0s ='+str(numzeros/len(p))
    
In [25]:
accurate(p,q)
accuracy = 1.0
ratio of 1s =0.16
ratio of 0s =0.84
In [28]:
def softmax(v):
    k = len(v)
    res = np.zeros(k)
    tot = 0.0
    for i in range(k):
        res[i] = np.exp(v[i])
        tot += res[i]
    for i in range(k):
        res[i] = res[i]/tot
    return res
In [29]:
def sigmoid(x):
    y = 1.0/(1.0+np.exp(-x))
    if y > 0.5:
        return 1
    else:
        return 0

if we wish, we may look at the trained W and b arrays.

In [37]:
w =W.eval()
In [38]:
w
Out[38]:
array([[  4.19662567e-03],
       [  5.16716051e+00],
       [ -5.04880219e+01]], dtype=float32)
In [39]:
lb =b.eval()
In [40]:
lb
Out[40]:
array([ 52.23804092], dtype=float32)
In [36]:
for i in range(20):
    print sigmoid(np.matmul(train_data[i], w)+lb)
0
0
1
0
0
0
1
0
0
0
0
1
1
0
1
0
0
0
0
1
/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:2: RuntimeWarning: overflow encountered in exp
  from ipykernel import kernelapp as app
In [ ]:
 
In [ ]:
 
In [ ]: