Cloud-init v. 0.7.7 running 'init-local' at Tue, 28 May 2019 20:06:43 +0000. Up 9.40 seconds. Cloud-init v. 0.7.7 running 'init' at Tue, 28 May 2019 20:06:44 +0000. Up 10.38 seconds. ci-info: ++++++++++++++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++++++++++++ ci-info: +--------+------+-----------------------------+---------------+-------+-------------------+ ci-info: | Device | Up | Address | Mask | Scope | Hw-Address | ci-info: +--------+------+-----------------------------+---------------+-------+-------------------+ ci-info: | eth0 | True | 172.31.46.177 | 255.255.240.0 | . | 0a:7c:4c:ae:b1:d0 | ci-info: | eth0 | True | fe80::87c:4cff:feae:b1d0/64 | . | link | 0a:7c:4c:ae:b1:d0 | ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | . | ci-info: | lo | True | ::1/128 | . | host | . | ci-info: +--------+------+-----------------------------+---------------+-------+-------------------+ ci-info: +++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++ ci-info: +-------+-------------+-------------+---------------+-----------+-------+ ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags | ci-info: +-------+-------------+-------------+---------------+-----------+-------+ ci-info: | 0 | 0.0.0.0 | 172.31.32.1 | 0.0.0.0 | eth0 | UG | ci-info: | 1 | 172.31.32.0 | 0.0.0.0 | 255.255.240.0 | eth0 | U | ci-info: +-------+-------------+-------------+---------------+-----------+-------+ Generating public/private rsa key pair. Your identification has been saved in /etc/ssh/ssh_host_rsa_key. Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub. The key fingerprint is: SHA256:k+w4e3HhZ7odiRPdpMKuYm1QF2ESZYfx994adfjT3Qg root@ip-172-31-46-177 The key's randomart image is: +---[RSA 2048]----+ | oo*+. | | +oo | | .. o | | ..oo. = o | | .So+.E o +| | .o.oo+oo =*| | oo.o++o o.B| | oo+..o . o.| | ..+. ... . | +----[SHA256]-----+ Generating public/private dsa key pair. Your identification has been saved in /etc/ssh/ssh_host_dsa_key. Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub. The key fingerprint is: SHA256:QzZRZxeYiThf1Tm5tavybvsXefjOwX33qApb67kKQhs root@ip-172-31-46-177 The key's randomart image is: +---[DSA 1024]----+ | .o..o=+oo| | o..o=. +o| | +o . =| | o .. o | | E S .o| | . o . o+o| | o . . . .+*| | . . + + o oB| | oo*o*=+o=| +----[SHA256]-----+ Generating public/private ecdsa key pair. Your identification has been saved in /etc/ssh/ssh_host_ecdsa_key. Your public key has been saved in /etc/ssh/ssh_host_ecdsa_key.pub. The key fingerprint is: SHA256:pimOIZzTc2C2kyvI+Fv8kMKrnFXqLXKl9YciY2Fimbk root@ip-172-31-46-177 The key's randomart image is: +---[ECDSA 256]---+ | | | | | | | | | ++ . S | |.B=+*+ + | |*+BO@.+ . | |+E=#*= o . | |.=X+=.o . | +----[SHA256]-----+ Generating public/private ed25519 key pair. Your identification has been saved in /etc/ssh/ssh_host_ed25519_key. Your public key has been saved in /etc/ssh/ssh_host_ed25519_key.pub. The key fingerprint is: SHA256:reGQvwUnYYbXfsd1Ku7Zb0G0hTFKbeKjI13gG3UgxPg root@ip-172-31-46-177 The key's randomart image is: +--[ED25519 256]--+ | +o oooo | | ...ooo.=o.| | . =o.+.+. =| | = +E +. =.| | o S.+=o.= | | +.*=o o . | | +.... .| | o . o .| | . o .o.| +----[SHA256]-----+ Generating locales (this might take a while)... Generation complete. Cloud-init v. 0.7.7 running 'modules:config' at Tue, 28 May 2019 20:06:51 +0000. Up 17.60 seconds. Cloud-init v. 0.7.7 running 'modules:final' at Tue, 28 May 2019 20:06:53 +0000. Up 18.95 seconds. Cloud-init v. 0.7.7 finished at Tue, 28 May 2019 20:06:53 +0000. Datasource DataSourceEc2. Up 19.12 seconds Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~16.04.1 running 'init-local' at Tue, 28 May 2019 20:10:59 +0000. Up 9.23 seconds. 2019-05-28 20:11:04,665 - util.py[WARNING]: Failed fetching dynamic/instance-identity from url http://169.254.169.254/2009-04-04/dynamic/instance-identity Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~16.04.1 running 'init' at Tue, 28 May 2019 20:11:06 +0000. Up 15.99 seconds. ci-info: ++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++ ci-info: +--------+------+-----------------------------+---------------+--------+-------------------+ ci-info: | Device | Up | Address | Mask | Scope | Hw-Address | ci-info: +--------+------+-----------------------------+---------------+--------+-------------------+ ci-info: | eth0 | True | 172.31.46.177 | 255.255.240.0 | global | 0a:7c:4c:ae:b1:d0 | ci-info: | eth0 | True | fe80::87c:4cff:feae:b1d0/64 | . | link | 0a:7c:4c:ae:b1:d0 | ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | host | . | ci-info: | lo | True | ::1/128 | . | host | . | ci-info: +--------+------+-----------------------------+---------------+--------+-------------------+ ci-info: +++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++ ci-info: +-------+-------------+-------------+---------------+-----------+-------+ ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags | ci-info: +-------+-------------+-------------+---------------+-----------+-------+ ci-info: | 0 | 0.0.0.0 | 172.31.32.1 | 0.0.0.0 | eth0 | UG | ci-info: | 1 | 172.31.32.0 | 0.0.0.0 | 255.255.240.0 | eth0 | U | ci-info: +-------+-------------+-------------+---------------+-----------+-------+ ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++ ci-info: +-------+-------------+---------+-----------+-------+ ci-info: | Route | Destination | Gateway | Interface | Flags | ci-info: +-------+-------------+---------+-----------+-------+ ci-info: | 0 | fe80::/64 | :: | eth0 | U | ci-info: | 3 | ff00::/8 | :: | eth0 | U | ci-info: +-------+-------------+---------+-----------+-------+ Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~16.04.1 running 'modules:config' at Tue, 28 May 2019 20:11:08 +0000. Up 18.11 seconds. Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~16.04.1 running 'modules:final' at Tue, 28 May 2019 20:11:09 +0000. Up 18.84 seconds. Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~16.04.1 finished at Tue, 28 May 2019 20:11:09 +0000. Datasource DataSourceEc2Local. Up 19.04 seconds Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~16.04.1 running 'init-local' at Sat, 26 Sep 2020 21:38:53 +0000. Up 16.97 seconds. Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~16.04.1 running 'init' at Sat, 26 Sep 2020 21:38:57 +0000. Up 20.63 seconds. ci-info: ++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++ ci-info: +--------+------+-----------------------------+---------------+--------+-------------------+ ci-info: | Device | Up | Address | Mask | Scope | Hw-Address | ci-info: +--------+------+-----------------------------+---------------+--------+-------------------+ ci-info: | eth0 | True | 172.31.20.181 | 255.255.240.0 | global | 06:a8:92:df:bf:60 | ci-info: | eth0 | True | fe80::4a8:92ff:fedf:bf60/64 | . | link | 06:a8:92:df:bf:60 | ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | host | . | ci-info: | lo | True | ::1/128 | . | host | . | ci-info: +--------+------+-----------------------------+---------------+--------+-------------------+ ci-info: +++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++ ci-info: +-------+-------------+-------------+---------------+-----------+-------+ ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags | ci-info: +-------+-------------+-------------+---------------+-----------+-------+ ci-info: | 0 | 0.0.0.0 | 172.31.16.1 | 0.0.0.0 | eth0 | UG | ci-info: | 1 | 172.31.16.0 | 0.0.0.0 | 255.255.240.0 | eth0 | U | ci-info: +-------+-------------+-------------+---------------+-----------+-------+ ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++ ci-info: +-------+-------------+---------+-----------+-------+ ci-info: | Route | Destination | Gateway | Interface | Flags | ci-info: +-------+-------------+---------+-----------+-------+ ci-info: | 0 | fe80::/64 | :: | eth0 | U | ci-info: | 4 | ff00::/8 | :: | eth0 | U | ci-info: +-------+-------------+---------+-----------+-------+ Generating public/private rsa key pair. Your identification has been saved in /etc/ssh/ssh_host_rsa_key. Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub. The key fingerprint is: SHA256:ZyRrvntFcCWuCELYhhsi0J2O/ALn6IoRwJbw2eqp9fQ root@ip-172-31-20-181 The key's randomart image is: +---[RSA 2048]----+ |+. =.. ... | |=.*+= .... | |o*o*o . . .o. | |+ =... . = .. | |.=.. S +. | |.oo.. o o . | |o +.. . . | |.= o . .. | |= . E oo | +----[SHA256]-----+ Generating public/private dsa key pair. Your identification has been saved in /etc/ssh/ssh_host_dsa_key. Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub. The key fingerprint is: SHA256:nhcW8GeqJ9d0htjbJ8UmK0whEqudbHFhHDBDk4jbM9U root@ip-172-31-20-181 The key's randomart image is: +---[DSA 1024]----+ | . oO=+. | | . . oXE. | | o .+ = + | | . ++ = O o . | | .oS = = + +| | o + * = = | | = + = + .| | = . o | | | +----[SHA256]-----+ Generating public/private ecdsa key pair. Your identification has been saved in /etc/ssh/ssh_host_ecdsa_key. Your public key has been saved in /etc/ssh/ssh_host_ecdsa_key.pub. The key fingerprint is: SHA256:Q4gN1rWKrfXVVhdi+uao7IwOeY1KcXY7GDORFEUnY4A root@ip-172-31-20-181 The key's randomart image is: +---[ECDSA 256]---+ | o..=** . o . | | . Eo.o.+ o . .| | . ooo . . .| | o o. ... . | | ..+*S.. oo | | o=.Oo..+ | | .+ +.+ . . | | . + + o | | ..o.= | +----[SHA256]-----+ Generating public/private ed25519 key pair. Your identification has been saved in /etc/ssh/ssh_host_ed25519_key. Your public key has been saved in /etc/ssh/ssh_host_ed25519_key.pub. The key fingerprint is: SHA256:fbtrLXQw09zfdRimKHpTAjySfejF+keNlUN7BT2OXOE root@ip-172-31-20-181 The key's randomart image is: +--[ED25519 256]--+ | + o ....+o| | o * + +.++.| | o * =o*=E.| | o o.= *++.+| | oS=. .+ =| | . + ..... o| | . o ..o | | o.. | | .oo | +----[SHA256]-----+ Cloud-init v. 19.1-1-gbaa47854-0ubuntu1~16.04.1 running 'modules:config' at Sat, 26 Sep 2020 21:39:05 +0000. Up 28.50 seconds. SOURCE: ======================================== #!/usr/bin/env python3 import os, sys, time, datetime, boto3, traceback from subprocess import check_call, check_output from os.path import expanduser, exists with open(__file__) as f: print("SOURCE:") print("="*40) print(f.read()) print("="*40) sys.stdout.flush() BUCKET = 'open-lambda-public' URL = 'https://s3.us-east-2.amazonaws.com/open-lambda-public/' s3 = boto3.client("s3") DATA = {'version': '0', 'id': '98f6d10c-cd88-98f6-3d26-79b057d83e1b', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'account': '667147198405', 'time': '2020-09-26T21:37:53Z', 'region': 'us-east-2', 'resources': ['arn:aws:events:us-east-2:667147198405:rule/ol-periodic-test'], 'detail': {}} def run(cmd): print("RUN "+cmd) check_call(cmd, shell=True, stdout=sys.stdout, stderr=sys.stderr) def aws_log(): with open("/var/log/cloud-init-output.log") as f: return f.read() def s3_put(key, value, content_type='text/plain'): print("S3 PUT "+key) r = s3.put_object(Bucket=BUCKET, Key=key, Body=value.encode('utf-8'), ContentType=content_type) def s3_get(key, default): try: r = s3.get_object(Bucket=BUCKET, Key=key) return str(r['Body'].read(), 'utf-8') except Exception as e: print(key, e) return default def s3_all_keys(Prefix): ls = s3.list_objects_v2(Bucket=BUCKET, Prefix=Prefix, MaxKeys=10000) keys = [] while True: contents = [obj['Key'] for obj in ls.get('Contents',[])] keys.extend(contents) if not 'NextContinuationToken' in ls: break ls = s3.list_objects_v2(Bucket=BUCKET, Prefix=Prefix, ContinuationToken=ls['NextContinuationToken'], MaxKeys=10000) return keys def run_all(): now = datetime.datetime.today().strftime("%Y-%m-%d-%H-%M-%S") dirname = 'vm/%s' % now # upload AWS log to S3 (we'll do again later to capture prints after this) s3_put(dirname+'/cloud-init-output.log', aws_log()) # move to user dir os.chdir(expanduser("~")) os.environ["HOME"] = expanduser("~") # why do we need to do this ourselves? # pull/build/test if not "commit" in DATA: run('git clone --depth=1 https://github.com/open-lambda/open-lambda.git') else: run(' && '.join([ 'git clone https://github.com/open-lambda/open-lambda.git', 'cd open-lambda', 'git checkout ' + DATA["commit"] ])) os.chdir("open-lambda") git_commit = check_output('git rev-parse HEAD', shell=True).decode('utf-8').strip() s3_put(dirname+'/commit.txt', git_commit) try: with open("build.out", "w") as f: check_call("make", shell=True, stdout=f, stderr=f) try: with open("tests.out", "w") as f: check_call("make test-all", shell=True, stdout=f, stderr=f) result = 'PASS' except Exception: print(traceback.format_exc()) result = 'FAIL' except Exception: print(traceback.format_exc()) result = 'BUILD-FAIL' # upload test results/logs to S3 s3_put(dirname+'/test.txt', result) if exists('build.out'): with open('build.out') as f: s3_put(dirname+'/build.out', f.read()) if exists('tests.out'): with open('tests.out') as f: s3_put(dirname+'/tests.out', f.read()) if exists('test.json'): with open('test.json') as f: s3_put(dirname+'/test.json', f.read()) if exists('testing/test-cluster/logs/worker-0.out'): with open("testing/test-cluster/logs/worker-0.out") as f: s3_put(dirname+'/worker-0.out', f.read()) # refresh summary gen_report() s3_put(dirname+'/cloud-init-output.log', aws_log()) # kill the VM (it has been configured to terminate on shutdown) if DATA.get("terminate", True): run('poweroff -f') def href(s3_path, all_keys): if not s3_path in all_keys: return '[NOT FOUND]' url = URL + s3_path return '{text}'.format(url=url, text=url.split("/")[-1]) def gen_report(): print("GEN REPORT") vms = set() all_keys = s3_all_keys("vm") for k in all_keys: vms.add(k.split('/')[1]) vms = sorted(vms, reverse=True) html = [] html += ['', ''] for i, vm in enumerate(vms): if i < 14: commit = s3_get('vm/%s/commit.txt'%vm, 'unkown').strip() result = s3_get('vm/%s/test.txt'%vm, 'unkown').strip() else: # just so we can complete faster without doing too many S3 reads commit = '???' result = '???' html += ['

%s [COMMIT: %s]

' % (vm, commit)] html += [''] html += ['', ''] html = '\n'.join(html) s3_put('tests.html', html, 'text/html') return html def main(): if len(sys.argv) == 1: run_all() elif sys.argv[1] == 'report': print(gen_report()) if __name__ == '__main__': main() ======================================== Cloning into 'open-lambda'... Checking out files: 4% (31/752) Checking out files: 5% (38/752) Checking out files: 6% (46/752) Checking out files: 7% (53/752) Checking out files: 8% (61/752) Checking out files: 9% (68/752) Checking out files: 10% (76/752) Checking out files: 11% (83/752) Checking out files: 12% (91/752) Checking out files: 13% (98/752) Checking out files: 14% (106/752) Checking out files: 15% (113/752) Checking out files: 16% (121/752) Checking out files: 17% (128/752) Checking out files: 18% (136/752) Checking out files: 19% (143/752) Checking out files: 20% (151/752) Checking out files: 21% (158/752) Checking out files: 22% (166/752) Checking out files: 23% (173/752) Checking out files: 24% (181/752) Checking out files: 25% (188/752) Checking out files: 26% (196/752) Checking out files: 27% (204/752) Checking out files: 28% (211/752) Checking out files: 29% (219/752) Checking out files: 30% (226/752) Checking out files: 31% (234/752) Checking out files: 32% (241/752) Checking out files: 33% (249/752) Checking out files: 34% (256/752) Checking out files: 35% (264/752) Checking out files: 36% (271/752) Checking out files: 37% (279/752) Checking out files: 38% (286/752) Checking out files: 39% (294/752) Checking out files: 40% (301/752) Checking out files: 41% (309/752) Checking out files: 42% (316/752) Checking out files: 43% (324/752) Checking out files: 44% (331/752) Checking out files: 45% (339/752) Checking out files: 46% (346/752) Checking out files: 47% (354/752) Checking out files: 48% (361/752) Checking out files: 49% (369/752) Checking out files: 50% (376/752) Checking out files: 51% (384/752) Checking out files: 52% (392/752) Checking out files: 53% (399/752) Checking out files: 54% (407/752) Checking out files: 55% (414/752) Checking out files: 56% (422/752) Checking out files: 57% (429/752) Checking out files: 58% (437/752) Checking out files: 59% (444/752) Checking out files: 60% (452/752) Checking out files: 61% (459/752) Checking out files: 62% (467/752) Checking out files: 63% (474/752) Checking out files: 64% (482/752) Checking out files: 65% (489/752) Checking out files: 66% (497/752) Checking out files: 67% (504/752) Checking out files: 68% (512/752) Checking out files: 69% (519/752) Checking out files: 70% (527/752) Checking out files: 71% (534/752) Checking out files: 72% (542/752) Checking out files: 73% (549/752) Checking out files: 74% (557/752) Checking out files: 75% (564/752) Checking out files: 76% (572/752) Checking out files: 77% (580/752) Checking out files: 78% (587/752) Checking out files: 79% (595/752) Checking out files: 80% (602/752) Checking out files: 81% (610/752) Checking out files: 82% (617/752) Checking out files: 83% (625/752) Checking out files: 84% (632/752) Checking out files: 85% (640/752) Checking out files: 86% (647/752) Checking out files: 87% (655/752) Checking out files: 88% (662/752) Checking out files: 89% (670/752) Checking out files: 90% (677/752) Checking out files: 91% (685/752) Checking out files: 92% (692/752) Checking out files: 93% (700/752) Checking out files: 94% (707/752) Checking out files: 95% (715/752) Checking out files: 96% (722/752) Checking out files: 97% (730/752) Checking out files: 98% (737/752) Checking out files: 99% (745/752) Checking out files: 100% (752/752) Checking out files: 100% (752/752), done.