Ipfs repo created by cli neither compatible with go-ipfs nor js-ipfs #16
Description
What I'm trying to accomplish:
I want to examine single ipfs objects created by orbit-db-cli with go-ipfs or js-ipfs. Given a <hash> from adding to a feed-store with orbitdb add <address> 'Hello World!', then I want to ipfs object get <hash> or jsipfs object get <hash> respectively.
This didn't work at all in the last couple of days, but now I'm at the point that I can make sense of the error messages.
What I think is wrong
The way orbit-db-cli sets up the ipfs repo is somewhat off, especially the /blocks subdirectory .
Take a look at my global ~/.ipfs repo created by go-ipfs init
~/.ipfs on machine
➜ ls blocks
000018.ldb BE CQ L5 QR RY
000020.ldb BG CR L7 QU S7
000021.log BI CS LA QV SA
23 BJ CURRENT LD QW SC
24 BN CV LG QX SD
25 BO CY LH R2 SE
26 BQ CZ LJ R3 SG
27 BS D2 LK R4 SH
2D BT D4 LN RB SHARDING
2E BV D6 LO RC SI
2F BZ D7 LOCK _README SJ
2G C7 DD LOG RF SK
2H CA DE LOG.old RG SM
2I CC DF LP RI SN
2L CF DH LS RJ SO
2M CG DL LT RN SR
2P CH DM LV RP SS
2W CI DO LZ RQ SU
2Z CJ DP M2 RS SV
32 CL DS M4 RV SW
35 CP DU MANIFEST-000019 RW SY
Now take a look at the ipfs repo created by orbit-cb-cli:
➜ ls ./orbitdb/ipfs/blocks
000005.ldb 000008.ldb 000009.log CURRENT LOCK LOG LOG.old MANIFEST-000007 _README SHARDINGAnd the following was created by a node script using the orbit-db module.
➜ ls ipfs_repo_by_node/blocks
7J D2 I7 IL K5 LG M4 OO PJ PU QP R3 _README S2 SHARDING TD X3 ZS
How I came to that conclusion
Examining the orbit-db-cli ipfs repo yields following results:
IPFS/experiments/local_orbitdb on machine
➜ IPFS_PATH=./orbitdb/ipfs ipfs object get QmRgyjgbNgDfxt3KQw4b1cM7ZZ9EgLdfmLd2h7m2gN5SVW
Error: required Datastore.Spec entry missing form config file
and
➜ IPFS_PATH=./orbitdb/ipfs jsipfs object get QmRgyjgbNgDfxt3KQw4b1cM7ZZ9EgLdfmLd2h7m2gN5SVW
/home/make/.config/yarn/global/node_modules/ipfs/src/cli/commands/object/get.js:15
throw err
^
Error: ENOENT: no such file or directory, open '/home/make/College/IPFS/experiments/local_orbitdb/orbitdb/ipfs/blocks/HC/CIQDDSBQ5HQ2AGXT3KOQVS5E3MN3LWYIBYP2F4BENYHJ2PWFKKBFHCI.data'
The js-ipfs implementation tells me, that there need to be the sharded directories and files under the /blocks directory, which is not the case when using orbit-db-cli.
If I use a node script to create an orbitdb database it creates the ipfs repo correctly with the sharded directories under ./<ipfs repo>/blocks and examining the ipfs objects works perfectly fine with jsipfs object get <hash>.
Notes
It seems that the go-ipfs implementation:
- Expects some config parameter that the current js implementation doesn't care about.
- Creates a directory called
/keystoreinstead of/keysunder./<ipfs repo>. (Has nothing to do with this issue, but maybe that'll be important latter? 😅 )
Whatever module creates the ipfs repo for orbit-db-cli it does not create it how it is expected.
If you use the tool ldb and look at the /blocks directory of orbit-db-cli ipfs repo you will find keys in leveldb what the /blocks directory should have looked like.
IPFS/experiments/local_orbitdb on machine
➜ cd orbitdb/ipfs/blocks
orbitdb/ipfs/blocks on machine
➜ ldb ./
>l
LIMIT set to 1000
>ls
/6T/CIQNNDI62MKDAM5ZVTK2TEINMYYU4HSC74RS3XZLBS2... /7J/CIQKKLBWAIBQZOIS5X7E32LQAL6236OUKZTMHPQSFIX...
/AL/CIQKXYLYLJ7WHZXF6NX6V45FWAS5U2LQ2K5CU2JWHXO... /FJ/CIQPWOVV72FZ65H6T6E4SONMKZB4NVBPNHNCHYB2U2Q...
/FR/CIQNTIIUTCQT6OSZS7OKCWEGWJLZW3KOWATDAID3PG4... /HC/CIQDDSBQ5HQ2AGXT3KOQVS5E3MN3LWYIBYP2F4BENYH...
/IC/CIQIU7KROHOI7JWOGFON3GBBT6PQDJI5AZP7BJG54M7... /IL/CIQJFGRQHQ45VCQLM7AJNF2GF5UHUAGGHC6LLAH6VYD...
/IV/CIQOWERFZ2KLTU2RN2D2CQKWCCLL3M3BNFRTL3UEENY... /K5/CIQPW4MAGTUNEBGZCEFZU7XAJL2BMIHGVB5ZR2IOKOS...
/LG/CIQJBQD2O6K4CGJVCCTJNUP57QHR4SKHZ74OIITBBGL... /M4/CIQOLBQZSZAODJGGH6RYYVBUXHTS3SM5EORZDU63LYP...
/OC/CIQOTRIFYF2I2WQAF6MLWIH3ASSUOU3PI644TXSTA5T... /OO/CIQBT4N7PS5IZ5IG2ZOUGKFK27IE33WKGJNDW2TY3LS...
/PD/CIQHWF2DR43OINVHEUB5NZ2BVZMCSFOPNPLP4XMATMT... /PJ/CIQB4F7VKKQDXHMXX6WYQZTRR5QVLP7VBQYAYW2Y5BA...
/PU/CIQJF2E6OPOEYYEVHYML4UD5A3R4QRAVBI7NVH3PL64... /QP/CIQNLGENZXNRUMUHZYGPPLZNZOMHHZVIU76LCD5GF5D...
/R3/CIQBED3K6YA5I3QQWLJOCHWXDRK5EXZQILBCKAPEDUJ... /S2/CIQPF3CHDB5GQ5ZBISOV2GWVMLAJPVEUMDMFKJZE7CM...
/SHARDING /X3/CIQFTFEEHEDF6KLBT32BFAGLXEZL4UWFNWM4LFTLMXQ...
/_README
>
Solution
So far I don't know what really causes this and how to fix it, but I might be able to find the cause of the problem on Saturday. I hope this helps anyone playing around with orbitdb. A workaround is to use node scripts to create orbitdbs and use js-ipfs to examine the ipfs objects.