Instructions to use Aobangaming/Aoban-3.0-220M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Aobangaming/Aoban-3.0-220M with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Aobangaming/Aoban-3.0-220M")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Aobangaming/Aoban-3.0-220M", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Aobangaming/Aoban-3.0-220M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Aobangaming/Aoban-3.0-220M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aobangaming/Aoban-3.0-220M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Aobangaming/Aoban-3.0-220M

SGLang

How to use Aobangaming/Aoban-3.0-220M with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Aobangaming/Aoban-3.0-220M" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aobangaming/Aoban-3.0-220M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Aobangaming/Aoban-3.0-220M" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Aobangaming/Aoban-3.0-220M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Aobangaming/Aoban-3.0-220M with Docker Model Runner:
```
docker model run hf.co/Aobangaming/Aoban-3.0-220M
```

Model Card for Model ID

Aoban 3.0 is a large, autoregressive transformer which utilizes FlashAttention and MHA.

Model Details

Model Description

Developed by: AobanZ
Model type: Transformer
Language(s) (NLP): English
License: MIT
Finetuned from model [optional]: Aoban-2.7-L

Model Sources [optional]

Repository: https://huggingface.co/Aobangaming/Aoban-3.0-220M

Uses

Aoban 3.0 is intended to be used for research, analysis and fine-tuning. It is not intended to be used for professional advice, or any kind of heavy work as generated outputs may be incorrect.

Direct Use

Aoban 3.0 can be used directly for text generation, experimentation, and conversational interactions. Users can provide text prompts and generate responses using the model's built-in language modeling capabilities.

Direct use is primarily intended for research and experimentation. Outputs may be incomplete, inaccurate, repetitive, or unrelated to the input, and should be evaluated before being used for other purposes.

Downstream Use [optional]

Aoban 3.0 may be fined-tuned for an AI Agents, Simple app helpers, and small conversational models. However, please note that generated outputs may be corrupted and/or incorrect.

Out-of-Scope Use

Heavy Work may overload the model, which will cause corrupted outputs and/or misinformation if implemented into a larger-app/ecosystem.

Bias, Risks, and Limitations

Aoban 3.0 is designed to process english and conversational text ONLY and cannot be fined-tuned for other uses.

Recommendations

We recommend users of Aoban 3.0 to finetune the model on new text, and add necessary guardrails and precautions to prevent misuse.

How to Get Started with the Model

Use the code below to get started with the model. The code will be added later. For now use the script 'ai thingy.py' to open an interactive interface in the python console.

Training Details

Training Data

Aoban 3.0 was trained on the full Conversational Fine Tuning dataset.

Training Procedure

Aoban 3.0 was trained on an RTX 3050 GPU, using FlashAttention and MHA.

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

Hyperparameter	Value	Comment
Precision	FP32
Optimizer	AdamW
Learning rate	5e-5	Inherited from Aoban 2.7
Batch size	16

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: RTX 3050 6GB
Hours used: 6
Cloud Provider: My Computer
Compute Region: Asia
Carbon Emitted: 0.21 kg CO₂e

Technical Specifications [optional]

Model Architecture and Objective

Aoban 3.0 is a casual decoder model which is autoregressive.

Hyperparameter	Value	Comment
Layers	16
D_MODEL	1024	Optimized for 64dim/head
Attention Heads	16
Vocabulary	~5600	w/ 160 Sequence length

Compute Infrastructure

Hardware

RTX 3050 6GB

Software

Windows 11, Intel i5-10400

Downloads last month: -; Downloads are not tracked for this model. How to track

Safetensors

Model size

0.2B params

Tensor type

F32

Model tree for Aobangaming/Aoban-3.0-220M

Unable to build the model tree, the base model loops to the model itself. Learn more.

Dataset used to train Aobangaming/Aoban-3.0-220M

Paper for Aobangaming/Aoban-3.0-220M

Quantifying the Carbon Emissions of Machine Learning

Paper • 1910.09700 • Published Oct 21, 2019 • 61