Home

Overview

pymport allows you to transparently use Python libraries from Node.js. It contains a fully self-contained embedded Python interpreter. It can also be built to use an existing external Python installation.

Two two interpreters share the same main thread and memory space. Python objects and V8 objects remain separate. V8 can access Python objects which have a PyObject type, while Python cannot access V8 objects. Python objects referenced from JavaScript must be freed by the V8 GC before being marked as available from the Python GC.

When passing objects between the interpreters:

• JavaScript to Python conversion is automatic and by copying
• Python objects are passed by reference to JavaScript as a PyObject, refer to Using PyObjects directly and Using proxified PyObjects
• There is a loose types equivalence between Python and JavaScript allowing to transform variables, refer to Types equivalence, including the section of passing functions and handling exceptions
• Inline Python is also supported through pyval

Functions also can be freely passed between the two interpreters. Every time a cross-language function is called, it is executed by the corresponding interpreter. Their arguments will be converted according to the same rules.

[New in 1.3] Full multi-threading safety is guaranteed and worker_threads are supported. Asynchronous calling of Python functions through callAsync in a background thread from the libuv pool is also supported.

• The Python interpreter is single-threaded and two threads cannot simultaneously run Python code. Well-written native extensions such as numpy will release the lock when running C/C++ code to allow for parallel execution. This means that if JavaScript calls asynchronously a Python function, all attempts at accessing a PyObject will block until that function has released the Python Global Interpreter Lock. This is a Python restriction that cannot be avoided.
• Unlike JavaScript, Python threads share the same memory storage - this means that worker_threads will share one Python environment. As Python supports multiple interpreter instances, this can eventually no longer be the case in a future version.

Importing user modules from the current directory

pymport is made for using Python libraries in Node.js. When importing Python modules, by default pymport will search only the library paths.

In order to import a user module from the current directory, or any other user directory, PYTHONPATH must be set accordingly before initializing Python. In CommonJS this can be set before the require:

process.env['PYTHONPATH'] = _dirname;
const { pymport } = require('pymport');

In TypeScript or ES6, there is no easy way to do this - in this case PYTHONPATH should be set from the environment.

Known Issues

• In 1.0 the V8 GC does not take into account the memory held by a PyObjects when deciding if they should be GCed or when the heap limit has been reached
• In 1.1 and later the V8 GC takes into account the memory held by a PyObject when it is initially referenced in JS but not its eventual growth after being referenced
• In 1.0 Python objects of type function never expire, so you will be leaking memory if you create Python lambdas in a loop (fixed in 1.1)
• #3, PyObjects are leaking memory in synchronous loops

Supported Versions

pymport is unit-tested on all combinations of:

Platforms	Windows x64, Linux x64 and macOS x64
Node.js	14.x, 16.x and 18.x
Python	3.8, 3.9, 3.10 and 3.11

Future Plans

• More features allowing direct interaction with PyObjects from JS
• (longer term) Asynchronous calling / Promises on the JS side vs multi-threading on the Python side
• (longer term) Generate TypeScript bindings from the Python modules
• (longer term) Using Node.js packages from Python, ie, an eventual jimport project, is currently blocked by PR#4352