To add a pure Python module to this app, add it to the site-packages directory. Make sure to add dependencies.
If a module contains C code, it's harder. It has to be embedded in the app as dynamic libraries.
NOTE: This text assumes the library you want to compile has only C, Cython or Python code. Scipy contains Fortran code, so it's harder to compile it.
The first step is compiling. To do so, you can use this tool. It's a command line tool to compile C projects to iOS. Clone it and install it.
$ git clone https://github.com/ColdGrub1384/compile_ios
...
$ cd compile_ios
$ ./install.sh
Then, download a release of the repository you want to compile, for example, Numpy. cd into the repo.
$ cd numpy-1.16.1
Then, run iosenv, this will open a shell with environment for compiling the library.
$ iosenv
...
To configure the repo, you may create a setup.cfg file with settings. For example, while compiling Matplotlib, that would be useful to not compile macOS support.
While compiling Numpy, you should set these environment variables to disable Blas, Lapack and Atlas: BLAS=None LAPACK=None ATLAS=None.
Run setup.py to build the extension.
$ python3 setup.py build_ext
...
Probably, many errors will be displayed. Check for the line where errors occurred and see if the code compiled because of a compilation condition. For example:
#ifdef CONDITION
code <- Error
#endifIn that case, try to undefine or set to 0 CONDITION.
#undef CONDITION
#ifdef CONDITION
code <- Error
#endifYou may also get errors like 'exc_[..]' undefined. Did you mean 'curexc_[..]'
In that case, just replace exc by curexc.
Make sure the extension compiled with no error, they can be hidden. You can redirect stdout to hide not useful output and just show errors and warnings.
When the extension compiled, see the content of the build/lib[..]. You should see some .so files.
Now, if you have many .so files, this will be hard. .so files cannot be directly embedded on the app bundle because the App Store will automatically reject that.
We have to make frameworks from those binaries.
Make a framework for each .so file.
Now, copy every .so file into its corresponding framework.
Then, on all Info.plist files inside frameworks, write this:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>en</string>
<key>CFBundleExecutable</key>
<string>[FILE NAME]</string>
<key>CFBundleIdentifier</key>
<string>[BUNDLE IDENTIFIER]</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>[BUNDLE NAME]</string>
<key>CFBundlePackageType</key>
<string>FMWK</string>
<key>CFBundleShortVersionString</key>
<string>1.0</string>
<key>CFBundleVersion</key>
<string>1</string>
<key>CFBundleSupportedPlatforms</key>
<array>
<string>iPhoneOS</string>
</array>
<key>MinimumOSVersion</key>
<string>12.0</string>
</dict>
</plist>NOTE: Here, by library name, I mean the name of the .so file and not the entire library.
Replace [FILE NAME] by the .so contained on the framework file name (include the extension), replace [BUNDLE NAME] by the name of the library, without extensions. And replace [BUNDLE IDENTIFIER] by the bundle identifier. For example: "com.yourcompany.libraryname". It cannot contain underscores.
Now you can embed the frameworks, without linking.
Run on device to check it works.
Now the module is embedded, this is the funny part. We have to import the module.
Open the main.m file under Pyto folder. Type this code:
[..]
PyMODINIT_FUNC (*PyInit__multiarray_umath)(void);
PyMODINIT_FUNC (*PyInit_fftpack_lite)(void);
PyMODINIT_FUNC (*PyInit__umath_linalg)(void);
PyMODINIT_FUNC (*PyInit_lapack_lite)(void);
PyMODINIT_FUNC (*PyInit_mtrand)(void);
void *_multiarray_umath = NULL; // _multiarray_umath.cpython-37m-darwin.so
void *fftpack_lite = NULL; // fftpack_lite.cpython-37m-darwin.so
void *umath_linalg = NULL; // umath_linalg.cpython-37m-darwin.so
void *lapack_lite = NULL; // lapack_lite.cpython-37m-darwin.so
void *mtrand = NULL; // mtrand.cpython-37m-darwin.so
void init_numpy() {
}
[..]
void init_python() {
[..]
// MARK: - Init builtins
#if MAIN
init_numpy();
[..]Replace "numpy" by the library you compiled name.
Each void pointer name corresponds to a .so file name. They represent libraries.
Each PyMODINIT_FUNC variable is a PyInit function from the C extension. Find the function by typing this:
$ nm -g [.so file] | grep PyInit
...
So, on the example, PyInit__multiarray_umath is a function from _multiarray_umath module.
They are uninitialized, so write this on the init function:
NSError *error;
for (NSURL *bundle in [NSFileManager.defaultManager contentsOfDirectoryAtURL:mainBundle().privateFrameworksURL includingPropertiesForKeys:NULL options:NSDirectoryEnumerationSkipsHiddenFiles error:&error]) {
NSURL *file = [bundle URLByAppendingPathComponent:[bundle.URLByDeletingPathExtension URLByAppendingPathExtension:@"cpython-37m-darwin.so"].lastPathComponent];
NSString *name = file.URLByDeletingPathExtension.URLByDeletingPathExtension.lastPathComponent;
void *handle;
if ([name isEqualToString:@"_multiarray_umath"]) {
load(_multiarray_umath);
} else if ([name isEqualToString:@"fftpack_lite"]) {
load(fftpack_lite);
} else if ([name isEqualToString:@"_umath_linalg"]) {
load(umath_linalg);
} else if ([name isEqualToString:@"lapack_lite"]) {
load(lapack_lite);
} else if ([name isEqualToString:@"mtrand"]) {
load(mtrand);
} else {
continue;
}
if (!handle) {
fprintf(stderr, "%s\n", dlerror());
}
}
*(void **) (&PyInit__multiarray_umath) = dlsym(_multiarray_umath, "PyInit__multiarray_umath");
*(void **) (&PyInit_fftpack_lite) = dlsym(fftpack_lite, "PyInit_fftpack_lite");
*(void **) (&PyInit__umath_linalg) = dlsym(umath_linalg, "PyInit__umath_linalg");
*(void **) (&PyInit_lapack_lite) = dlsym(lapack_lite, "PyInit_lapack_lite");
*(void **) (&PyInit_mtrand) = dlsym(mtrand, "PyInit_mtrand");
PyImport_AppendInittab("__numpy_core__multiarray_umath", PyInit__multiarray_umath);
PyImport_AppendInittab("__numpy_fft_fftpack_lite", PyInit_fftpack_lite);
PyImport_AppendInittab("__numpy_linalg__umath_linalg", PyInit__umath_linalg);
PyImport_AppendInittab("__numpy_linalg_lapack_lite", PyInit_lapack_lite);
PyImport_AppendInittab("__numpy_random_mtrand", PyInit_mtrand);This code looks for all frameworks and finds libraries used by Numpy. For each C extension, write:
else if ([name isEqualToString:@"<LIBRARY NAME>"]) {
load(<LIBRARY NAME>);Then, initialize all PyInit functions:
*(void **) (&<NAME OF PyInit FUNCTION>) = dlsym(<C EXTENSION>, "<NAME OF PYINIT FUNCTION>");Now, the most important thing:
PyImport_AppendInittab("__numpy_core__multiarray_umath", PyInit__multiarray_umath);__numpy_core__multiarray_umath corresponds to numpy.core._multiarray_umath. Replace values with yours.
cd into the library repo and build the entire library (iosenv isn't needed).
$ cd numpy-1.16.1
$ python3 setup.py build
...
Then, remove all .so files from the build.
$ find build -name "*.so" -delete
Copy the folder under build/lib* to site-packages.
Run the app and try to import your libraries. Errors will be displayed. Try to look paths of libraries that failed to import.
For example: numpy.core._multiarray_umath. Then write:
PyImport_AppendInittab("__numpy_core__multiarray_umath", PyInit__multiarray_umath);Open site-packages/extensionsimporter.py. Before # MARK: - All, write this:
[..]
# MARK: - NumPy
class NumpyImporter(object):
"""
Meta path for importing NumPy to be added to `sys.meta_path`.
"""
__is_importing__ = False
def find_module(self, fullname, mpath=None):
if fullname in ('numpy.core._multiarray_umath', 'numpy.fft.fftpack_lite', 'numpy.linalg._umath_linalg', 'numpy.linalg.lapack_lite', 'numpy.random.mtrand'):
return self
if fullname == 'numpy' and not self.__is_importing__:
return self
return
def load_module(self, fullname):
f = fullname
if f != 'numpy':
f = '__' + fullname.replace('.', '_')
mod = sys.modules.get(f)
if mod is None:
def importMod():
mod = importlib.__import__(f)
sys.modules[fullname] = mod
if fullname != 'numpy' or __host__ is widget:
importMod()
else:
try:
self.__is_importing__ = True
importMod()
self.__is_importing__ = False
except KeyboardInterrupt:
pass
except SystemExit:
pass
except Exception as e:
print(e)
report_error('Numpy', traceback.format_exc())
raise
finally:
self.__is_importing__ = False
return mod
return mod
[..]
# MARK: - All
__all__ = ['NumpyImporter', 'MatplotlibImporter', 'PandasImporter'] # Add here the name of the function you created.Replace "Numpy" by the name of the library you're trying to add.
Replace
('numpy.core._multiarray_umath', 'numpy.fft.fftpack_lite', 'numpy.linalg._umath_linalg', 'numpy.linalg.lapack_lite', 'numpy.random.mtrand')by the name of the builtin C extensions the library will import. Include the name of the library.
Open Startup.py from Xcode project on Pyto > Resources > Startup.py.
Then, add your importer function name here:
for importer in (NumpyImporter, MatplotlibImporter, PandasImporter):