Rodolfo Mendes

Author: Rodolfo Mendes

  • How to Run Local AI Models with Ollama and Save Money

    How to Run Local AI Models with Ollama and Save Money

    A few days ago, I posted a tutorial in the Spring AI Dev community on how to use Anthropic Claude’s models with Spring AI and received a curious comment from a user:

    Why does it make it sound like you can only use Amazon or Anthropic?

    In my reply, I explained that, for most companies, trusting proprietary solutions makes sense. Usually, you do not have time to build your own solution, especially if AI is not the focus of your business. On the other hand, you have the money and such strong demand that you can negotiate better prices. Sounds fair, right?

    However, I still could not get that question out of my head. Weren’t there any more options? And why should I spend money just to experiment with a tutorial? 

    Well, this article is a response to that question!

    Fortunately, running models locally offers a practical way to bypass these costs and take full control of your data. Enter Ollama.

    It does not mean you can just get rid of proprietary AI. Your business will likely integrate with a proprietary AI provider such as OpenAI, Anthropic, or AWS Bedrock.

    But in fact, it does not make sense to spend money if you are still learning AI, and Ollama is an excellent option to fill that gap.

    What is Ollama?

    Ollama is an AI server that you can install in your computer and then run different AI models locally. Then you can connect harnesses and agents such as Open Code or Open Claw, or build your AI solution with a framework like Spring AI. Everything runs on your hardware, and no data leaves your computer.

    With proprietary AI providers like OpenAI and Anthropic, you need to register for a subscription or buy credits in advance so you can connect to their API’s. And in many cases, if you want to use a more advanced model, you have to buy extra credits or upgrade to a premium subscription.

    Also, proprietary AI providers host their models in their dedicated infrastructure. Their models do not leave their servers, so you need to transmit your data for processing.

    With Ollama, you do not need to spend money to use an AI, and your prompts never leave your computer; everything runs locally. Thus, besides saving money, Ollama is a solution to be considered in production if your company or organization has severe privacy requirements and no data could be sent to third-party services.

    Understanding the Trade-Off

    Obviously, running AI models locally does not come for free. Because you are running these models locally, your choices are limited by your hardware. Cutting-edge models like deepseek-r1:671b have more than 600 billion parameters and require 400 GB of VRAM to run, which is not feasible for most people. 

    But don’t worry. Smaller models with 8 billion parameters are also available and run in a common graphics card, providing decent results.   

    Keeping these hardware boundaries in mind, let’s look at how to check your system specs and select the right model for your setup.

    So, now that we understand what Ollama is and its tradeoffs, let’s jump to the practical part of this article. In the next section, we will learn how to install Ollama and run local models. 

    How to Install Ollama and Run Local Models

    In this example, I installed Ollama on in Ubuntu 24.04.3 LTS instance running on a WSL2 system. The official documentation provides additional instructions for other operating systems such as Windows and macOS. The Linux installation does not rely on dependency managers like apt, so these instructions are also portable to other distributions. 

    1. Installing Ollama

    Ollama installation is quite simple. As long as you have the curl command installed, all you have to do is download and run the installation script. First, make sure that curl is installed. Run the command:

    curl -V

    And observe the result. You should receive a positive message listing the version, libraries, and protocols. If you do not have curl installed, just run the following command to install it:

    sudo apt install curl -y

    After curl is installed, run the command below:

    curl -fsSL https://ollama.com/install.sh | sh

    This command will download the installation script and install it in your system. Then you can confirm the installation with the following command:

    ollama help

    This command will print instructions for using the Ollama runner, including a list of available commands for administering and running models.

    If the script executed correctly, it must have configured Ollama as a Linux service, and you can administer it with the systemctl command. For example, you can check the status of the ollama service with the command:

    systemctl status ollama

    And then the systemctl command will print whether the service is active, along with additional information such as the process ID, memory usage, and CPU usage.   

    If you received the usage instructions from the ollama help command and verified that the service is active with systemctl, this means the installation was successful. We can proceed to run local models.

    Now that the background service is up and running, we can verify everything works by communicating directly with its local API.

    2. Running Local Models

    If you installed Ollama correctly, you can now run commands in your terminal and call the HTTP API Ollama provides on port 11434 by default. You can test the API by running the command:

    
curl http://localhost:11434


    And receive the message “Ollama is running”.

    But that does not mean you can generate text from prompts. For example, try to run the following command:

    curl http://localhost:11434/api/generate -d '{'
    "model": "gemma3",
    "prompt": "Why is the sky blue?"
}'

    And you’ll get a 404 error: “model gemma3 not found” 

    And that’s because Ollama is just a vessel, like the Infinity Gauntlet. The true power of the Infinity Gauntlet comes from the Infinity Stones that you attach to the gauntlet. Ollama is the same thing; its true power comes from the multiple models that you can download and install.

    There are two ways you can verify the models installed on your Ollama instance. The first one is the list command. In your terminal, type:

    ollama list

    The command will print a table with the models available for use in your local installation. If you run the command immediately after installing Ollama, you will only see the table headers. That means you have no model locally available yet. 

    Another way to check the available models is via the /tags/endpoint. Run the command:

    curl http://localhost:11434/api/tags

    And you will receive a JSON object with an empty “models” array. Confirming that no model is available yet. 

    Now that we have Ollama installed and running, let’s learn how to install a model locally in our environment.

    3. Choosing a Model

    Ollama provides an extensive library of models available at https://ollama.com/library.

    When visiting the Ollama library, you’ll notice many models available in different sizes and capabilities, and choosing the best one can be a hard task. Model benchmarks are way beyond the scope of this humble article, but here are some tips.

    First, think about the application you are building and what capabilities you need. For example, if your application needs to integrate with external API’s, then you need tooling. If you need to interpret images, then you need a multi-modal model, and so on.

    As said before, you are limited by your hardware. If you are not sure, you can run the following command to discover how much memory you have, and therefore, the largest model your hardware supports:

    nvidia-smi --query-gpu=name,memory.total,memory.used,memory.free,driver_version --format=csv

    In my case, I got the following output:

    name, memory.total [MiB], memory.used [MiB], memory.free [MiB], driver_version
NVIDIA GeForce RTX 5070 Laptop GPU, 8151 MiB, 3052 MiB, 4840 MiB, 610.88

    Which means I have a hard limit of 8GB of VRAM, which is the maximum size of a model I can run locally.

    For this example, I’m choosing a multimodal model – capable of processing image and text – which fits entirely in my VRAM. Given these requirements, gemma3 seems a reasonable choice. Let’s see how to pull and run the model locally.

    For this example, I’m choosing a multimodal model – capable of processing image and text – which fits entirely in my VRAM. Given these requirements, gemma3 seems a reasonable choice. Let’s see how to pull and run the model locally.

    4. Pulling a Model from the Registry

    Fortunately, the analogy to Infinite Stones ends here, and you don’t have to throw someone you love off a cliff to get a model. Instead, all you have to do is run a simple command:
    ollama pull gemma3

    The pull command will download and install the model in your local environment. If Ollama downloaded the model correctly, then you will see the model listed if you run the Ollama list command or if you access the tags endpoint through the HTTP API. Run the following commands and check that the gemma3 model is now available in your local environment

    ollama list

    Or

    curl http://localhost:11434/api/tags

    And they should return the gemma3 model.

    5. Running a Model Locally

    After downloading the model, you can test it in an interactive session in the terminal with the command:

    ollama run gemma3

    With this command, you can chat with the model right from the terminal, or even use it in a command pipeline. Also, you can now submit prompts via the Web API at either the /api/chat or /api/generate endpoint. Run the command:

    curl http://localhost:11434/api/generate -d '{
     "model": "gemma3",
     "stream": false,
     "prompt": "Why is the sky blue?"
}'

    And have a nice conversation about our atmosphere.

    6. Integrating with Spring AI (Optional)

    Finally, if you are a Spring AI developer like me, connect your Spring AI application to your local Ollama instance. Configure the base URL in your application.yml file:

    spring:
    ai:
        ollama:
            base-url: http://localhost:11434

    Conclusion

    Congratulations! Now you can run AI models in your local environment and save on API credits or subscriptions.

    Now it is up to you to explore the possibilities. You can test different models and see which one provides the best answers for your tasks, or connect your preferred harness to use local models.   Also, you can use these local models to develop your own AI solutions with Spring AI, and this is the possibility that excites me the most.

    References

  • Building a Machine Learning Model: Collecting Data

    Data is the main ingredient of Machine Learning models. An actual Machine Learning project starts with the team assessing if the necessary data is available.

    If not (which would be no surprise), the team needs to develop a strategy to collect and store the necessary data before modeling Machine Learning Algorithms.

    Common sources of data include:

    • Databases
    • Computer files
    • Websites
    • REST APIs
    • Sensor data
    • Physical files
    • Satellite images
    • User interaction in apps
    • Videos

    Most Machine Learning algorithms are agnostic to the ingested type of data: most of them receive matrixes and numeric arrays as parameters. From the learning algorithms perspective, it does not matter if the feature came from ebooks or images.

    Thus, in Machine Learning Project, a good part of the effort comes from setting the correct strategy to collect, store, clean, and preprocess the data, making them beneficial to train Machine Learning models. 

  • Types of Machine Learning: Deep Learning

    Deep Learning is a branch of Machine Learning specialized in Artificial Neural Networks with multiple intermediary layers. 

    Neural Networks with multiple layers can approximate very complex non-linear functions. That’s why Deep Learning has been very successful in CV (Computer Visions), NLP (Natural Language Processing), and Reinforcement Learning. 

    Deep Learning architectures include:

    • Convolutional Neural Networks
    • Recurrent Neural Networks 

    Due to its ability to learn high-order non-linear functions, Deep Learning algorithms can easily overfit small datasets. Also, Deep Learning algorithms may be computationally expensive, even for a small amount of data.

    Thus, Deep Learning is not a silver bullet. Deep Learning can overfit easily with no gain to generalize the solution for simple problems over a small dataset, making it a poor choice for this domain.

    However, Deep Learning has achieved stunning results for complex problems that benefit from large amounts of data, longer training time, and intense computational power. 

  • Types of Machine Learning: Time Series Analysis

    The typical regression task predicts the value of a target variable based on the values of one or more feature variables. For example, to predict the price of a house based on its characteristics like size, number of rooms, etc.

    But in some cases, we want to predict the value of a variable based on its past values. In our example, we would predict the price of a house based on its previous prices instead of its characteristics.

    We call this modeling Time Series Analysis. A Time Series is a collection of observations – values of a variable – ordered along the time. The time elapsed between observations varies years, months, days, seconds, or even milliseconds.

    A simple model for Time Series Analysis is moving averages (MA). This model assumes that the following observation is the average of all past observations.

    Another model for Time Series Analysis is the autoregressive model (AR). This model assumes that the following observation is linear dependent on the last p values of the time series. In other words, it’s a linear regression model applied to the p previous observations.

    Finally, from combining these models, we form the more general models autoregressive-moving-averages (ARMA) and the autoregressive-integrated-moving-averages model (ARIMA).

  • Types of Machine Learning: Recommender Systems

    Recommender systems are a category of Machine Learning algorithms that predict a user’s rating or preferences over a collection of items. 

    Recommender systems algorithms include:

    • Collaborative filtering
    • Content-based filtering
    • Session-based
    • Session-based recommender systems 

    Reinforcement learning for recommender systems include:

    • Multi-criteria recommender systems
    • Risk-aware recommender systems
    • Mobile recommender systems 

  • Types of Machine Learning: Reinforcement Learning

    In the reinforcement learning paradigm, the learning process is a loop in which the agent reads the state of the environment and then executes an action.

    Then the environment returns its new state and a reward signal, indicating if the action was correct or not. The process continues until the environment reaches a terminal condition or it reaches a maximum number of iterations.

    These are some of the main concepts in Reinforcement Learning:

    The Environment

    The environment is a representation of the context that our agent will interact with. It can represent an aspect of the natural world, like the stock market, or a street, or a completely virtual environment, like a game.

    State

    States are observations that the agent receives from the environment. It’s the way the agent gets all available information about the environment.

    Actions

    Actions are performed by the agent and may change the state of the environment. All the rules of how an action changes the state of the environment are internal to the environment. For a given state, the agent can choose its following action, but it does not control how this action will affect the environment.

    Rewards

    Rewards signal to the agent if an action was correct or not.

    So, if you want to learn more about Reinforcement Learning by a practical example, take a look at the following blog post:

    reinforcement-learning4.fun/2019/06/09/int…

  • Machine Learning Applications: Topic Discovery with Clustering

    Given a set of documents, a common task is to group them accordingly to topics or subjects. A human agent can create a hierarchy of subjects and assign each document to its related issue.

    However, a clustering algorithm can create this structure automatically and more precisely. We can apply hierarchical clustering algorithms to group documents as we build the hierarchy between them.

    We don’t need to set the subjects nor the topic hierarchy previously. Instead, the clustering process discovers them.

    However, human agents need to provide some parameters to the algorithms:

    • a criterion to measure the similarity between documents (a distance function)
    • a level to cut the hierarchy or minimum size of a cluster
    • labels for the discovered clusters

  • Exploratory Data Analysis for Machine Learning

    Exploratory data analysis is the most challenging task when building a machine learning model, especially for beginners.

    A result of the No-Free-Lunch-Theorem is that there’s no single model that will perform well for every dataset. In other words, there’s no silver bullet Machine Learning Algorithm.

    The practical consequence is that we need to make a LOT of human decisions when building our model: which algorithm to use, which features to use, which features to discard, apply normalization, regularization, hyperparameters to tune.

    And because the space of decisions is so vast, going on simple trial-and-error is a shot in the dark. We need to drive our decisions on actions that could potentially benefit our model.

    So, the only way to make better decisions when building a model is to understand our dataset. And that’s why an excellent Exploratory Data Analysis is an essential step in Machine Learning model building.

  • Machine Learning Applications: Stroke Prediction

    The Stroke Prediction Dataset at Kaggle is an example of how to use Machine Learning for disease prediction.

    The dataset comprises more than 5,000 observations of 12 attributes representing patients’ clinical conditions like heart disease, hypertension, glucose, smoking, etc. For each instance, there’s also a binary target variable indicating if a patient had a stroke.

    We can build a model to predict the occurrence of a stroke by training typical classification algorithms, for example, Logistic Regression, K-Nearest Neighbors, Support Vector Machines classifiers, Decision Trees, or others.

    Of course, the actual applicability of such a model depends on how representative the patients dataset is. However, this is a good exercise for those who wish to understand how to apply machine learning in healthcare. 

    Reference:

    https://www.kaggle.com/fedesoriano/stroke-prediction-dataset

  • Machine Learning Applications: Medical Diagnosis

    An increasing application of machine learning classification algorithms is medical diagnose. Diagnosing if a patient has a specific disease is a simple binary classification problem.

    For example, we may build a model to identify if a patient has Hepatitis C. In this case, there are two possible outputs: yes and no, which is a typical binary classification problem.

    Thus, the big challenge for medical diagnosis is collecting the correct data to train a model for a specific disease: blood tests, x-ray images, ultrasound, etc.

    Each of these data types can help diagnose and need particular treatment before used to feed ML models.

    Finally, we’re far from a general artificial intelligence system capable of replacing doctors. However, the excellent results obtained by applying ML models to clinical data show that ML will be a powerful tool for healthcare.