Using CoreML on macOS

[JonasPfeifer05]

Hello, I have a MacBook with a M4 chip which comes with a neural engine. But when using fastembed, the neural engine is not used at all, instead the cpu is running on 100%.

Is there a way to tell fastembed to use the neural engine?

Or is this feature just missing?

[JonasPfeifer05]

Monitoring during running:

[Anush008]

Hey 👋. Can you try using

fastembed-rs/src/text_embedding/init.rs

Lines 47 to 54 in cfec7d7

	/// Set the execution providers for the model
	pub fn with_execution_providers(
	mut self,
	execution_providers: Vec<ExecutionProviderDispatch>,
	) -> Self {
	self.execution_providers = execution_providers;
	self
	}

?

Ref: https://ort.pyke.io/perf/execution-providers for CoreML

[aim-bot-mtl]

Hey @Anush008 !!

Do you think that adding "default execution providers" can be a good idea ?

Something like this:

.with_execution_providers({
	#[cfg(any(target_os = "macos", target_os = "ios"))]
	{
		use ort::{CoreMLExecutionProvider, XNNPACKExecutionProvider};

		[
			CoreMLExecutionProvider::default().build(),
			XNNPACKExecutionProvider::default().build(),
		]
	}

	#[cfg(target_os = "windows")]
	{
		use ort::DirectMLExecutionProvider;

		[DirectMLExecutionProvider::default().build()]
	}

	...
})

If yes, I'll try smtg this week and PR it ;) Cool lib and cool resource to learn basic things with ort btw ! :)

Not my main gh account, thats why it looks weird lol (@PierreLouisLetoquart)

[Anush008]

I believe this would be better left to the users to choose.

[aim-bot-mtl]

Okay yes thats what I thought because of the large model choice