Skip to main content

Installation

Cortex.cpp offers four installer types​

  • Network Installers download a minimal script and require an internet connection to fetch packages during installation.
  • Local Installers include all necessary packages, enabling offline installation without internet access.
  • Dockerfile Installers are used to build a Docker image with Cortex ready to go.
  • Binary files without package management.

Cortex.cpp supports three channels​

  • Stable: The latest stable release on github.
  • Beta: The release candidate for the next stable release, available on github release with the tag vx.y.z-rc1
  • Nightly: The nightly build of the latest commit on dev branch, available on discord.

For more information, please check out the different channels.

Download URLs​

TypeVersionOperating SystemDownload Link
LocalStableWindowsDownload
LocalStableLinuxDownload
LocalStableMacOSDownload
info

For other versions, please look at cortex.cpp repo or each installer page.

Minimum Requirements​

OS​

  • MacOS 12 or later
  • Windows 10 or later
  • Linux: Ubuntu 20.04 or later, Debian 11 or later, and any of the latest versions of Arch (for other distributions, please use the Dockerfile installer or binary files, we have not tested on other distributions yet.)

Hardware​

CPU​

info
  • Cortex.cpp supports a processor that can handle AVX2. For the full list, please see here.
  • We support older processors with AVX, AVX-512 and non-AVX, though this is not recommended.
Intel CPU​
  • Haswell processors (Q2 2013) and newer.
  • Tiger Lake (Q3 2020) and newer for Celeron and Pentium processors.
AMD CPU​
  • Excavator processors (Q2 2015) and newer.

RAM​

info

We support DDR2 RAM as the minimum requirement but recommend using newer generations of RAM for improved performance.

  • 8GB for running up to 3B models (int4).
  • 16GB for running up to 7B models (int4).
  • 32GB for running up to 13B models (int4).

GPU​

info

Having at least 6GB VRAM when using NVIDIA, AMD, or Intel Arc GPUs is recommended.

  • 6GB can load the 3B model (int4) with ngl at 120 ~ full speed on CPU/ GPU.
  • 8GB can load the 7B model (int4) with ngl at 120 ~ full speed on CPU/ GPU.
  • 12GB can load the 13B model (int4) with ngl at 120 ~ full speed on CPU/ GPU.
info
  • NVIDIA driver version 470.63.01 or higher. Note: Users must install the driver by themselves.
  • CUDA Toolkit version 12.0 or higher. Note: Cortex.cpp will automatically detect and install the required version of cudart to the user's machine.

Disk​

  • At least 10GB of free space for downloading models.

Different channels​

Different channels have different features, stability levels, binary file name, app and data folders.

Stable​

  • App name: cortexcpp
  • Binary file name: cortex, cortex-server for linux and mac; cortex.exe, cortex-server.exe for windows.;
  • App folder (Windows Only): C:\Users\<username>\AppData\Local\cortexcpp
  • Data folder: ~/cortexcpp for linux and mac; C:\Users\<username>\cortexcpp for windows.
  • Uninstaller script (Mac Only): cortex-uninstall.sh

Beta​

  • App name: cortexcpp-beta
  • Binary file name: cortex-beta, cortex-server-beta for linux and mac; cortex-beta.exe, cortex-server-beta.exe for windows.;
  • App folder (Windows Only): C:\Users\<username>\AppData\Local\cortexcpp-beta
  • Data folder: ~/cortexcpp-beta for linux and mac; C:\Users\<username>\cortexcpp-beta for windows.
  • Uninstaller script (Mac Only): cortex-beta-uninstall.sh

Nightly​

  • App name: cortexcpp-nightly
  • Binary file name: cortex-nightly, cortex-server-nightly for linux and mac; cortex-nightly.exe, cortex-server-nightly.exe for windows.;
  • App folder (Windows Only): C:\Users\<username>\AppData\Local\cortexcpp-nightly
  • Data folder: ~/cortexcpp-nightly for linux and mac; C:\Users\<username>\cortexcpp-nightly for windows.
  • Uninstaller script (Mac Only): cortex-nightly-uninstall.sh