Merge branch 'master' of https://github.com/aws-samples/amazon-fsx-tutorial

Darryl Osborne · Darryl Osborne · commit 7b4ae48ae35f · 2020-05-12T11:02:57.000-06:00
diff --git a/lustre/03-load-data/readme.adoc b/lustre/03-load-data/readme.adoc
@@ -128,7 +128,7 @@ time tree --du -h /mnt/fsx/CMIP5
 +
 ----
 threads=36
-time lfs find /mnt/fsx/CMIP5--type f | parallel --will-cite -j ${threads} sudo lfs hsm_restore {}
+time lfs find /mnt/fsx/CMIP5 --type f | parallel --will-cite -j ${threads} sudo lfs hsm_restore {}
 
 ----
 . *_Monitor_* the network throughput of the instance from the other SSH terminal window running `*nload*`. These objects are not copied through the instance from S3 to the file system. The GET API is called in parallel to copy data directly from S3 to the OSTs.
diff --git a/lustre/fsx-ior-read-benchmark.sh b/lustre/fsx-ior-read-benchmark.sh
@@ -22,11 +22,33 @@ sudo mkdir -p /fsx
 sudo mount -t lustre -o noatime,flock <fsid>.fsx.<region>.amazonaws.com@tcp:/<mountname> /fsx 
 sudo chown ec2-user:ec2-user /fsx
 
-# IOR write - from one instance
-mpirun --npernode 4 --oversubscribe ior --posix.odirect -t 1m -b 1320m -g -v -w -i 1 -F -k -D 0 -o /fsx/ior.bin
 
-# IOR read - from all instances
-mpirun --npernode 4 --oversubscribe ior --posix.odirect -t 1m -g -r -i 1000000 -F -k -D 0 -z -o /fsx/ior.bin
+
+## write test
+
+# IOR - write test from all instances
+instance_id=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
+mpirun --npernode 8 --oversubscribe ior --posix.odirect -t 1m -b 1m -s 16384 -g -v -w -i 100 -F -k -D 0 -o /fsx/ior-${instance_id}.bin
+
+
+
+## read test
+
+# IOR - write test from one instance
+mpirun --npernode 8 --oversubscribe ior --posix.odirect -t 1m -b 1m -s 8192 -g -v -w -i 1 -F -k -D 0 -o /fsx/ior.bin
+
+# IOR - read test from all instances
+mpirun --npernode 8 --oversubscribe ior --posix.odirect -t 1m -b 1m -s 8192 -g -r -i 10000 -F -k -D 60 -z -o /fsx/ior.bin
+
+
+
+## in-memory cache read test
+
+# IOR - write test from one instance
+mpirun --npernode 8 --oversubscribe ior --posix.odirect -t 1m -b 1m -s 675 -g -v -w -i 1 -F -k -D 0 -o /fsx/ior.bin
+
+# IOR - read test from all instances
+mpirun --npernode 8 --oversubscribe ior --posix.odirect -t 1m -b 1m -g -r -i 2000000 -F -k -D 0 -z -o /fsx/ior.bin
 
 
 

Original file line number	Diff line number	Diff line change
`@@ -128,7 +128,7 @@ time tree --du -h /mnt/fsx/CMIP5`
`128`	`128`	`+`
`129`	`129`	`----`
`130`	`130`	`threads=36`
`131`		`-time lfs find /mnt/fsx/CMIP5--type f \| parallel --will-cite -j ${threads} sudo lfs hsm_restore {}`
	`131`	`+time lfs find /mnt/fsx/CMIP5 --type f \| parallel --will-cite -j ${threads} sudo lfs hsm_restore {}`
`132`	`132`
`133`	`133`	`----`
`134`	`134`	. _Monitor_ the network throughput of the instance from the other SSH terminal window running `nload`. These objects are not copied through the instance from S3 to the file system. The GET API is called in parallel to copy data directly from S3 to the OSTs.