Getting started with LLMs - Ollama is a good starting point
Daniel Nashed – 24 November 2024 19:49:40
llama.cpp is one of the most GitHub projects, which is used for many integrations https://github.com/ggerganov/llama.cpp.
But it is also quite complex to run, because you need to download LLMs.
Once you looked into other projects, you can really appreciate how much easier it can be to use Ollama.
https://ollama.com/
It's available for Linux, Mac and Windows.Ollama takes away all the complexity of compiling optimized for the target CPU and GPU.
And even more important: It automatically downloads and runs LLMs for you, without any conversion.
Ollama provides a registry for downloading LLMs.
For Windows and Mac there is a simple download. For Linux there is a simple convenience script to download.
Once installed you can download and start the LLM via
ollama run llama3.2
That's really all you need to get started. Anyone who downloaded a LLM manually and you can start querying the LLM.
The server also listens to 127.0.0.1:11434 with an OpenAI compatible endpoint.
>>> what is the meaning of 42?
The number 42 has become a cultural phenomenon due to its appearance in Douglas Adams' science fiction series "The Hitchhiker's Guide to the Galaxy." In the book, a supercomputer named Deep Thought is asked to find the "Answer to the Ultimate Question of Life, the Universe, and Everything." After 7.5 million years of computation, Deep Thought finally reveals that the answer is... 42.
However, the book also notes that the characters don't actually know what the ultimate question is, making the answer essentially meaningless without context. This paradoxical situation has led to various interpretations and
memes surrounding the number 42.
Some possible meanings or interpretations of 42 include:
1. **The Answer**: In a humorous sense, 42 can be seen as the ultimate answer to life's mysteries.
2. **Absurdity**: The number 42 represents the absurdity and randomness of the universe, highlighting the limitations of human understanding.
3. **Symbolism**: Some see 42 as a symbolic representation of the impermanence and fleeting nature of existence.
4. **Mathematical curiosity**: For mathematicians, 42 is an interesting number that has no special properties or significance beyond its numerical value.
Ultimately, the meaning of 42 is subjective and open to individual interpretation. It has become a cultural icon, symbolizing the search for answers to life's big questions, and often used humorously to represent "the answer" to
any question.
List LLMs
ollama list
NAME ID SIZE MODIFIED
llama3.2:latest a80c4f17acd5 2.0 GB 44 hours ago
Help command-line
Here are the command line options. It's pretty simple.
ollama
Usage:
ollama [flags]
ollama [command]
Available Commands:
serve Start ollama
create Create a model from a Modelfile
show Show information for a model
run Run a model
stop Stop a running model
pull Pull a model from a registry
push Push a model to a registry
list List models
ps List running models
cp Copy a model
rm Remove a model
help Help about any command
Infos about models
To get details about a model, there is a useful command including the quantization.
That's specially important for lower performing GPUs.
ollama show llama3.2
Model
architecture llama
parameters 3.2B
context length 131072
embedding length 3072
quantization Q4_K_M
Parameters
stop "<|start_header_id|>"
stop "<|end_header_id|>"
stop "<|eot_id|>"
License
LLAMA 3.2 COMMUNITY LICENSE AGREEMENT
Llama 3.2 Version Release Date: September 25, 2024
- Comments [0]
![[HCL Domino]](../lotus-domino.gif){kind=small}
![[Domino on Linux]](../domino-linux.gif){kind=small}
![[Nash!Com]](../nashcom.gif){kind=small}
![[Daniel Nashed]](../daniel-nsh.gif){kind=small}