SI: Implement put operations#67

Merged

susodapop merged 41 commits intosifrom

Dec 30, 2022

Commits on Nov 15, 2022

Basic PUT operation. Currently this never executes because the server
Jesse Whitehouse
committed
Bump Spark CLI service protocol version being used.
Jesse Whitehouse
committed
Log when attempting a staging operation
Jesse Whitehouse
committed

Commits on Nov 16, 2022

Fix failing unit tests since function signature for ExecuteResponse changed
Jesse Whitehouse
committed
Add e2e test for put.
Jesse Whitehouse
committed
Bail on tests if staging_ingestion_user is not set
Jesse Whitehouse
committed
Black client.py
Jesse Whitehouse
committed

Commits on Nov 17, 2022

Add unit test that sanity checks _handle_staging_operation is called
Jesse Whitehouse
committed
Fix imports so that this module can be run independently:
Jesse Whitehouse
committed
Implement GET operation
Jesse Whitehouse
committed

Commits on Nov 23, 2022

Refactor client.py into distinct methods for each ingestion command type
Jesse Whitehouse
committed
Update pypoetry so I can develop on Python 3.10
Jesse Whitehouse
committed
Applied PR feedback around explicit response codes.
Jesse Whitehouse
committed
Applying PR feedback
Jesse Whitehouse
committed
PR feedback
Jesse Whitehouse
committed
Black client.py
Jesse Whitehouse
committed
Refactor e2e test to use a single teste for PUT, GET, and REMOVE
Jesse Whitehouse
committed
Make REMOVE command work
Jesse Whitehouse
committed
These methods don't need to know the `operation`
Jesse Whitehouse
committed
Remove single quote that broke query
Jesse Whitehouse
committed
Remove unneeded argument
Jesse Whitehouse
committed
Expect operation to succeed
Jesse Whitehouse
committed
Black PySQLStagingIngestionTestSuite only
Jesse Whitehouse
committed
Tidy up comments in e2e test
Jesse Whitehouse
committed
Basic e2e test scaffolded in. Currently fails.
Jesse Whitehouse
committed
Only allow ingestion commands when base_uploads_path is specified
Jesse Whitehouse
committed
Restrict local file operations to descendents of uploads_base_path
Jesse Whitehouse
committed

Commits on Dec 20, 2022

Remove per PR feedback
Jesse Whitehouse
committed
Add check for null local_file per PR feedback
Jesse Whitehouse
committed
Open output stream _after_ successful HTTP request
Jesse Whitehouse
committed
Resolve relative paths before comparing row.localFile to uploads_base_path
Jesse Whitehouse
committed
Add test that PUT fails if file exists in staging location and OVERWRITE not set
Jesse Whitehouse
committed
Add tests: operations fail to modify another user's staging location
Jesse Whitehouse
committed
Add test that ingestion command fails if local file is blank
Jesse Whitehouse
committed
Add test that invalid staging path will fail at server
Jesse Whitehouse
committed

Commits on Dec 22, 2022

Basic usage example (needs tweaking)
Jesse Whitehouse
committed
Add samples of GET and REMOVE
Jesse Whitehouse
committed

Commits on Dec 28, 2022

Refactor to allow uploads_base_path to be either a single string object
Jesse Whitehouse
committed

Commits on Dec 29, 2022

Refactor uploads_base_path to staging_allowed_local_path
Jesse Whitehouse
committed

Commits on Dec 30, 2022

Fix mypy static type failures
Jesse Whitehouse
committed
Black src files
Jesse Whitehouse
committed