Out of memory when running script

Technical Discussion
1

Hi @mliberty,
I am currently working with accumulators.py for long-term current logging.
Using Joulescope JS110 on windows 10 with 64bit processor, 16gb ram installed.
When executing the script i had faced an error multiple times.

11:06:57 ,12.7 C ,10.4 µA ,3.538138573834052 mAh
FATAL: src/backend/windows.c:214 out of memory
FATAL: src/backend/windows.c:105 mutex lock 'heap' failed
FATAL: src/backend/windows.c:105 mutex lock 'heap' failed

Can i you please provide the solution to this.
Thanks

2

Hi @aryan - I am not sure why your computer is running out of memory, which is what that error is telling you. I changed the title for this topic from “Mutex lock ‘heap’ failed error”, which is just a consequence of out of memory.

I need a lot more information to be of help here.

1. What versions of software are you using:

a. Python:

> python -VV
Python 3.12.4 (tags/v3.12.4:8e8a4ba, Jun  6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)]

b. Joulescope packages:

> pip show joulescope pyjoulescope_driver
Name: joulescope
Version: 1.1.15
Summary: Joulescope™ host driver and utilities
Home-page: https://joulescope.readthedocs.io
Author: Jetperch LLC
Author-email: joulescope-dev@jetperch.com
License: Apache 2.0
Location: C:\bin\Python3_12_4\Lib\site-packages
Requires: numpy, psutil, pyjls, pyjoulescope-driver, pymonocypher, python-dateutil, pywin32
Required-by:
---
Name: pyjoulescope_driver
Version: 1.5.4
Summary: Joulescope™ driver
Home-page: https://joulescope.readthedocs.io
Author: Jetperch LLC
Author-email: joulescope-dev@jetperch.com
License: Apache 2.0
Location: C:\bin\Python3_12_4\Lib\site-packages
Requires: numpy, psutil, pywin32, requests
Required-by: joulescope

If you are not running joulescope 1.1.15 and pyjoulescope_driver 1.5.4 as I show above, use this command to update:

pip install -U pyjoulescope_driver joulescope

2. What code are you running?

Are you running the most recent, unmodified accumulators.py example? The output from this script is different from what you show, so I suspect you are running something else:

> python .\pyjoulescope_examples\bin\accumulators.py
Accumulation in progress.
Updates displayed every 60 seconds.
Press CTRL-C to stop.

duration = 60 s, charge = -159 nC, energy = -95.9 nJ
duration = 120 s, charge = -315 nC, energy = -191 nJ
duration = 180 s, charge = -463 nC, energy = -285 nJ
duration = 240 s, charge = -603 nC, energy = -374 nJ
duration = 300 s, charge = -741 nC, energy = -463 nJ
duration = 360 s, charge = -883 nC, energy = -556 nJ
duration = 420 s, charge = -1.03 µC, energy = -653 nJ
duration = 480 s, charge = -1.17 µC, energy = -749 nJ
duration = 540 s, charge = -1.30 µC, energy = -841 nJ
duration = 600 s, charge = -1.44 µC, energy = -937 nJ
duration = 660 s, charge = -1.60 µC, energy = -1.04 µJ

I just ran this with a JS110, and I do not see any memory growth over time using Task Manager that would indicate a memory leak.

Do you you have custom or modified code? If so, do you see this same issue when you run the unmodified accumulators.py?

3. When does this error occur?

a. How long after starting this script does the error occur? Right away? Many hours? The output should show you almost exactly.

b. Are you doing anything else on your computer when this error occurs? Be sure to also note antivirus scans, backups, etc.

4. Is your computer keeping up?

Your computer may not have enough CPU to keep up with whatever you are asking it to do. In this case, pyjoulescope_driver can sometimes try to allocate more memory to store sample updates that are never serviced in python due to processing starvation. One way to significantly reduce the compution requirements is to provide the --source sensor argument. This tells the script to use the statistics computed on the JS110 rather than streaming all samples to the host.

> python .\pyjoulescope_examples\bin\accumulators.py --source sensor
3

Hi @mliberty ,
I was running exe for accumulators.py with few changes like:

  1. I have printed energy in mAh also, by formula from Coulombs
  2. Some prints have been disabled
  3. Printing interval is set to 10seconds
  4. Used Logging module for logs.
    As i was executing this exe with above mentioned changes when i faced these errors.
    The timing for this error occurrence was not anything particular or size of logs was just random.

Also,
I am getting error of:
WARNING pkt_index skip: expected 41059, received 41114
WARNING pkt_index skip: expected 41434, received 41572
WARNING pkt_index skip: expected 42340, received 42752
this error has also not occurred after particular time,
once this occurred just when starting the exe,
second time it occurred after almost 65hours.

4

Hi @aryan,

If I understand correctly, you are running custom code (modified accumulators.py). I suspect that one of your modifications is causing a memory leak. You should try seeing if the unmodified accumulators.py works correctly for you.

The warnings for pkt_index skip indicate that your host computer is not keeping up with sample streaming. This usually introduces only a very small error due to missing samples. On Windows, this is often caused by update checks, antivirus scans, and backups which can cause the entire Windows OS to become unresponsive. You can avoid this problem almost entirely by using on-instrument statistics as I mentioned in (4).

5

Thanks @mliberty,
will update you shortly.

Regards