LangChain and node-llama-cpp

[DarkSorrow]

Hello,

I was trying to check how to use the embedding with LangChain and it seems that i have to initialise my model twice.
I saw that on the normal package i could use the create context but i can't access it with the exemple i saw

import { LlamaCpp } from "@langchain/community/llms/llama_cpp"; // "@langchain/community": "^0.3.22",
import { LlamaCppEmbeddings } from "@langchain/community/embeddings/llama_cpp";

const llm = await LlamaCpp.initialize({
  modelPath,
  temperature: 0.7,
  verbose: true,
});

const embeddings = await LlamaCppEmbeddings.initialize({
  modelPath,
});

Am i missing something to be able to use embedding or is there another way to initialise things?

[DarkSorrow]

Seems the bindtools that is suppose to be linked to functions are not there.

const calculatorSchema = z.object({
  location: z.string().describe("The location to get the weather for."),
  unit: z.enum(["celsius", "fahrenheit"]).describe("The unit of temperature to return."),
});
const weatherTool = tool(
  async (input: { location: string; unit: "celsius" | "fahrenheit" }): Promise<string> => {
    return `The current weather in ${input.location} is 20 degrees ${input.unit}.`;
  }, {
    name: "get_current_weather", 
    description: "Get the current weather in a given location",
    schema: calculatorSchema,
});

const weatherLLM = llm.bindTools([weatherTool]); // bind tools does not exist in LlamaCpp

I understand that the library is probably not complete yet as you are already working a lot on the current one. I'll try to see if i can start checking myself and maybe contribute because this is linked to the functions of the current library right?
I know you are already pretty busy with this library and i'll just finish by saying its already amazing what you guys did with this lib already :)

[giladgd]

We didn't implement the LangChain integration, but I have plans to significantly improve it in the future.
I plan to add a few more big features to node-llama-cpp before I do so, though.

Using a dedicated model for embedding instead of the same large model you use for inference might work better and be more efficient for you, as embedding models are more optimized for this task, which makes them smaller, faster, and consume less memory.