All Questions
16 questions
5
votes
1
answer
115
views
Periodically syncing data from an API to database is extremely slow
I have a scheduled task where I sync all the data from one table (via an API) to another. There are about 10,000 rows that are being updated and the new ones get created. This is what I came up with ...
- 255
1
vote
0
answers
55
views
Listing all related details of a model object through Django ORM
How can my solution be improved / optimized?
explaination of db models :
thinkpad is a equipment of laptop subcategory.and ...
- 127
2
votes
0
answers
73
views
PySide2 Row and Column Creator
In short, I need help simplifying and optimizing this code because now that I have switched to a tree widget from 2 list widgets, loading the tables has increased by roughly 3-4 seconds (used to be ...
- 21
1
vote
1
answer
77
views
Code improvement suggestions for cost calculation [closed]
I have to calculate costs based on country code and city(optional). There is also a condition that countries in Asia have separate pricing logic depending on both country and city and they are stored ...
- 255
1
vote
1
answer
113
views
My Python 3.6 script to detect stamp card redeem fraud and prepare a csv report
I have a requirement to write a Python 3.6 script which should be able to detect following case within a time period and be able to create a text file report that lists the condition violated and ...
- 21
4
votes
2
answers
92
views
Pulling log messages from a MySQL table
I want to get this code reviewed for a function I wrote that pulls log messages from a MySQL table:
...
- 870
3
votes
0
answers
229
views
Database check in a loop
I have a python script that loads a database table and checks an API for the entries found.
I do it like this:
...
- 131
4
votes
2
answers
1k
views
Loop through database and run shell commands with Python and exiftool
Briefly, I'm looking at getting the code below to execute faster. I have 100k images to go through. I'm running a query against MySQL, looping through results and then running exiftool against an ...
- 380
5
votes
1
answer
957
views
Inserting products scraped from the web into a database
I am trying to speed up the runtime of my program as the API can make calls. However after adding the inserts to a database 3 mins turns in 7 mins after 180 API calls, which can be one a second. How ...
- 163
3
votes
1
answer
129
views
Fetching settings for batches of wafers using loops
I have this code below that needs to query multiple database tables.
...
- 243
2
votes
2
answers
2k
views
Database migration script
I'm working on a database migration script written in python, which will take data from a MySQL database and insert them into a PostgreSQL database with a different Schema (different table structures, ...
- 145
7
votes
1
answer
16k
views
Efficient Pandas to MySQL "UPDATE... WHERE"
I have a pandas DataFrame and a (MySQL) database with the same columns. The database is not managed by me.
I want to update the values in the database in an "UPDATE... WHERE" style, updating only ...
- 171
3
votes
2
answers
2k
views
Django ListView with MySQL query for the queryset
(Note: Even if you don't know Django, it's probably the query that needs work anyway, so you probably don't need to know Django/Python)
I have a ListView that ...
- 247
7
votes
1
answer
4k
views
How to optimize a pipline for storing item in mysql?
I am using scrapy framework for data scraping and dumping item in MySQL database.
Here is my pipeline that is inserting output to MySQL, but its taking so much time. Any suggestions on how to ...
- 243
5
votes
6
answers
948
views
