Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

alt

Daniel Nashed

Getting started with LLMs - Ollama is a good starting point

Daniel Nashed – 24 November 2024 19:49:40

llama.cpp
is one of the most GitHub projects, which is used for many integrations
https://github.com/ggerganov/llama.cpp.
But it is also quite complex to run, because you need to download LLMs.
Once you looked into other projects, you can really appreciate how much easier it can be to use Ollama.


https://ollama.com/

It's available for Linux, Mac and Windows.Ollama takes away all the complexity of compiling optimized for the target CPU and GPU.
And even more important: It automatically downloads and runs LLMs for you, without any conversion.


Ollama provides a registry for downloading LLMs.

For Windows and Mac there is a simple download. For Linux there is a simple convenience script to download.



Image:Getting started with LLMs - Ollama is a good starting point

Once installed you can download and start the LLM via


ollama run llama3.2



That's really all you need to get started. Anyone who downloaded a LLM manually and you can start querying the LLM.

The server also listens to 127.0.0.1:11434 with an OpenAI compatible endpoint.



>>>
what is the meaning of 42?
The number 42 has become a cultural phenomenon due to its appearance in Douglas Adams' science fiction series "The Hitchhiker's Guide to the Galaxy." In the book, a supercomputer named Deep Thought is asked to find the "Answer to the Ultimate Question of Life, the Universe, and Everything." After 7.5 million years of computation, Deep Thought finally reveals that the answer is... 42.


However, the book also notes that the characters don't actually know what the ultimate question is, making the answer essentially meaningless without context. This paradoxical situation has led to various interpretations and

memes surrounding the number 42.


Some possible meanings or interpretations of 42 include:


1. **The Answer**: In a humorous sense, 42 can be seen as the ultimate answer to life's mysteries.

2. **Absurdity**: The number 42 represents the absurdity and randomness of the universe, highlighting the limitations of human understanding.

3. **Symbolism**: Some see 42 as a symbolic representation of the impermanence and fleeting nature of existence.

4. **Mathematical curiosity**: For mathematicians, 42 is an interesting number that has no special properties or significance beyond its numerical value.


Ultimately, the meaning of 42 is subjective and open to individual interpretation. It has become a cultural icon, symbolizing the search for answers to life's big questions, and often used humorously to represent "the answer" to

any question.



List LLMs


ollama list

NAME               ID              SIZE      MODIFIED

llama3.2:latest    a80c4f17acd5    2.0 GB    44 hours ago



Help command-line


Here are the command line options. It's pretty simple.


ollama

Usage:

ollama [flags]

ollama [command]


Available Commands:

serve       Start ollama

create      Create a model from a Modelfile

show        Show information for a model

run         Run a model

stop        Stop a running model

pull        Pull a model from a registry

push        Push a model to a registry

list        List models

ps          List running models

cp          Copy a model

rm          Remove a model

help        Help about any command



Infos about models


To get details about a model, there is a useful command including the quantization.
That's specially important for lower performing GPUs.


ollama show llama3.2

Model

  architecture        llama

  parameters          3.2B

  context length      131072

  embedding length    3072

  quantization        Q4_K_M


Parameters

  stop    "<|start_header_id|>"

  stop    "<|end_header_id|>"

  stop    "<|eot_id|>"


License

  LLAMA 3.2 COMMUNITY LICENSE AGREEMENT

  Llama 3.2 Version Release Date: September 25, 2024



Comments
No Comments Found

Links

    Archives


    • [HCL Domino]
    • [Domino on Linux]
    • [Nash!Com]
    • [Daniel Nashed]