I am trying to export joulescope current and voltage values to an excel file every 20 ms when the sleep current of a device goes above 1 mA. When I run this, the data is added to excel about twice/second because of the amount of index skips that are happening. I am having trouble resolving this. This is in python and I believe the block that is causing the delayed data output is
with device:
device_data = device.read(contiguous_duration=0.001)
current, voltage = device_data[-1, :]
If I put this block outside the while loop, the data outputs into excel every 20 ms, but the values are all the same.
while True:
with device:
device_data = device.read(contiguous_duration=0.001)
current, voltage = device_data[-1, :]
# Check if the current is less than 1mA
if current < 0.001:
continue
# Breaks data readings when it reaches one hour
elif time.time() - start_time >= max_sample_time_excel:
continue
else:
if current > current_high_threshold:
counter_to_cause_data_logging += 1
try:
# Try to load existing Excel file
workbook = openpyxl.load_workbook(file_name)
worksheet = workbook.active
row = worksheet.max_row + 1
print(f"Loaded existing worksheet with {row-1} rows")
except FileNotFoundError:
# Create new Excel file if it doesn't exist
workbook = openpyxl.Workbook()
worksheet = workbook.active
worksheet.append(['Count', 'Current A', 'Voltage V'])
row = 2
print("Created new worksheet")
if counter_to_cause_data_logging >= consecutive_high_threshold:
# Append the current and voltage value to the worksheet
worksheet.cell(row=row, column=1, value=row-1)
worksheet.cell(row=row, column=2, value=current)
worksheet.cell(row=row, column=3, value=voltage)
row += 1
# Save the worksheet to the Excel file
workbook.save(file_name)
# Wait for 20 ms before taking another reading
time.sleep(0.02)
else:
time.sleep(1)
I am not sure that I completely understand what you are trying to do. The device.read function is definitely not instantaneous as it starts sample streaming, waits for contiguous_data (which can take a while with the JS110 and some PCs), captures the date, and stops streaming. This will miss huge real-time gaps, much, much greater than 20 milliseconds.
However, Joulescopes support continuous sample streaming. I think that you want to start streaming and then inspect the incoming samples as they arrive. If some condition occurs, then you log the data. You repeat this process until done, then stop streaming.
Here are some questions to better understand what you are trying to do:
What is your desired threshold condition to log a window? Do you want a single full-rate sample exceeding 1 mA? The average over a 20 ms window to exceed 1 mA? Some downsampled 20 ms data (50 Hz) sample to exceed a threshold?
What do you really want to log? Just one sample? The average value over the 20 ms window? The 50 Hz downsampled value? Something else?
What do you plan to do with this data? I am not a fan of using Excel to log sample data. Excel only supports 1 million rows, which the JS110 can fill up in 1/2 seconds at full-rate.
Good morning, sir. Apologizes for the confusing question.
1 and 2: If the sleep current of our device ever goes above 1 mA, I want to list a current and voltage value every 20 ms (1 excel row every 20 ms) until the current dips back under 1 mA or until an hour is reached. This should result in around 180,000 rows. Here are some specifications I was given.
Sample every 1s, test current is < 1mA
If current > 1mA for 2 straight samples (for longer than a full second at least), trigger faster readings
Sample every 20ms, continue to read until current drops < 1mA, or for 1hr maximum
Save the file with the timestamp and date - exported to Excel
3: The reason we want to use excel is because the excel files are much smaller than a jls file. If we record a data point every 20 ms for up to an hour max, I don’t think we would ever get close to 1 million rows. We would use this data to see any trends in the high sleep current.
4: JS110
5: 64-bit Windows 10
Let me know if I can clarify in any other way. I believe the part when you talked about continuous sample streaming could be the issue. Using continuous instead of contiguous_data.
Hi @Marf - If I understand correctly, you are just trying to implement a trigger to save on filesize. What if you simply record 50 Hz current and voltage data to a JLS file? The estimated file size is only 1.5 MB / hour.
The Joulescope UI 0.10.x has a problem with recording highly downsampled signals (see this discussion). However, you can use the Joulescope UI 1.0.x alpha, which should be great for this purpose.
Unless you want to automate the capture or perform automated analysis, it’s likely much easier to use the Joulescope UI to capture all the data and then view it later.
You want to configure only “i” and “v” inputs, along with 50 Hz, like this:
Again, I apologize for the lack of clarification. Here is the full description of what I am trying to accomplish.
I am creating a long term script that will essentially run non-stop for months-years. We want to run our device non-stop to see if or when a defect occurs. Specifically, a defect occurs when the sleep current goes above 1 mA. Here are some parameters.
If we hit any values that are above 1 mA, trigger a 60s JLS file to pull in high sampling frequency data
Then, after this first 60s, start saving current and voltage values at a slower 20ms interval for up to 60min, or until the current goes under 1 mA
If the time limit is reached, a txt file will then take over and record how much longer after the initial hour the defect goes on for to prevent all excel rows from getting filled up
My issue is only coming from #2. Back to your response, I am familiar with the Joulescope UI and have used it for smaller applications. However, I am trying to automate capturing current/voltage values when a threshold is reached over long time periods.
Hi @Marf! Yes, long-term sample streaming with conditions to record data to files is certainly possible. From what you are describing, reliability and stability will be crucial.
I recommend using our new pyjoulescope_driver directly without the joulescope wrapper or older v0 python backend. The advantage with the pyjoulescope_driver is that it performs USB communication outside of the Python GIL, which means you have fewer interactions with other blocking operations in your Python script, such as file write access.
I create a new example, monitor. This initial version configures the statistics output for 50 Hz. At 50 Hz, the value of each statistic is computed over 20 ms windows. When the current average exceeds 1 mA, we start logging these statistics. I also included a feature to add the previous second to the log to help troubleshoot the lead-up to the event. This code logs to both Excel and JLS v2 so that you can try both and see what you like.
For now, this example does not implement your most recent request to also log full-rate data for the first 60 seconds. Everything is configured and ready to be added, but I am out of time for now.
Take a look. Let me know how it works for you and if it makes sense!
Good afternoon, sir. I was able to get the pyjoulescope_driver installed with the requirements.txt. However, when I run your monitor version, I am unable to figure out a ModuleNotFoundError. The script cannot find a module named ‘pyjoulescope_driver.binding’. Is this something on my end?
Hi @Marf - Can you be more specific about how you installed pyjoulescope_driver? Normally, requirements.txt is not part of installing a package from pypi. Instead, you would do this:
Good morning. I downloaded the ‘joulescope_driver-main’ from Github onto my desktop and did the correct pip installations. I put your ‘monitor’ version into that folder and attempted to run it. This could be my problem. Where am I supposed to compile ‘monitor’ from? Also, am I supposed to replace any files with new files from ‘joulescope_driver-main’?
Ah, ok. I developed monitor.py with a JS220. I did not have access to a JS110 while traveling last week. I just tested with a JS110, and I see the same error.
I just updated monitor.py 75fddc5 to correctly support the JS110. Download it again or git pull, and it should work for you.
That seems to have fixed it. I am receiving OverflowErrors, but I believe I will be able to correct this unless you have any ideas. Regardless, thank you for the help!
For reference, I plugged in a PCB with an average sleep current of 8 mA to force data logging. The only thing that prints is a 30 second jls file, then proceeds with the errors listed below.
Hi @Marf - have you modified monitor.py? It looks like the error that occurs on line 127 should be line 158 in the version I provided.
Also, you need to use 64-bit python. You can check which version you are using with this command:
python -VV
Note that is two “V”. While you can use any python >= 3.8, I recommend 3.10.10 since it is the most tested currently. You can download python for Windows here.
No, no code has been modified. The difference in the lines were some enters getting left out. I fixed that. Also, I have 64-bit python in version 3.10.10
I connected a JS110, and then wired a JS220 Evaluation Kit 1 to the JS110 inputs to provide a constant ~10 mA:
evk.off()
evk.power_ldo(1.0)
evk.resistance = 100
I then run monitor.py. After a while, I pressed CTRL-C to end the capture. I then double-clicked on the JLS file in Windows File Explorer to load it into Joulescope UI 0.10.13:
The new Joulescope UI 1.0.x does not seem to want to load it due to only a single UTC time entry. This will be fixed soon.
So, this monitor.py script works successfully for me. We need to figure out what is different between my computer and your computer. Can you ensure that the python version on the path is 3.10.10? Please post the output of python -VV:
