Gradio

Welcome to the Leaderboard for open Hebrew LLMs. The leaderboard ranks the different models according to their success on various tasks on Hebrew.

The leaderboard was created and is operated by a collaboration of Mafat / The Israeli National Program for NLP in Hebrew and Arabic and DICTA: The Israel Center for Text Analysis.

ברוכים הבאים ללוח התוצאות של מודלי LLM פתוחים בעברית. לוח התוצאות מדרג את המודלים השונים לפי הצלחתם במשימות שונות בעברית.

לוח התוצאות נוצר ומתופעל על ידי שיתוף פעולה בין מפא"ת / התוכנית הלאומית הישראלית ל-NLP בעברית ובערבית ודיקטה: המרכז הישראלי לניתוח טקסטים


⭕	mistralai/Mistral-Large-Instruct-2407	73.62	90.95	74.65	75.5	88.49	34.73	77.41	122.61	mistralai/Mistral-Large-Instruct-2407
⭕	ai21labs/AI21-Jamba-Large-1.6	72.57	93.1	76.44	68.2	81.65	35.98	80.07	398.56	ai21labs/AI21-Jamba-Large-1.6
🟢	meta-llama/Meta-Llama-3.1-70B	72.14	88.57	77.72	77.5	80.94	34.7	73.42	70.55	meta-llama/Meta-Llama-3.1-70B
⭕	CohereForAI/c4ai-command-r-plus-08-2024	71.58	91.67	77.75	67.83	85.25	36.85	70.1	103.81	CohereForAI/c4ai-command-r-plus-08-2024
🟢	meta-llama/Meta-Llama-3-70B	71.18	87.86	77.31	76.7	81.29	34.47	69.44	70.55	meta-llama/Meta-Llama-3-70B
⭕	CohereForAI/c4ai-command-r-plus	70.84	91.67	77.17	65.9	84.53	36	69.77	103.81	CohereForAI/c4ai-command-r-plus
🟢	google/gemma-3-27b-pt	69.38	85.24	77.53	64.97	81.65	36.45	70.43	27.43	google/gemma-3-27b-pt
⭕	CohereForAI/aya-expanse-32b	68.96	92.14	75.74	67.97	82.73	34.7	60.47	32.3	CohereForAI/aya-expanse-32b
🟢	meta-llama/Llama-4-Scout-17B-16E	68.54	90	77.78	67.83	79.14	35.34	61.13	108.64	meta-llama/Llama-4-Scout-17B-16E
🟦	meta-llama/Llama-3.3-70B-Instruct	68.51	77.14	72.24	77.33	81.65	33.91	68.77	70.55	meta-llama/Llama-3.3-70B-Instruct
⭕	nvidia/Llama-3.1-Nemotron-70B-Instruct-HF	67.95	77.86	70.71	75.5	83.81	33.39	66.45	70.55	nvidia/Llama-3.1-Nemotron-70B-Instruct-HF
🟢	Qwen/Qwen2.5-72B	67.92	95.48	73.44	71.5	74.82	32.79	59.47	72.71	Qwen/Qwen2.5-72B
⭕	meta-llama/Llama-3.1-70B-Instruct	67.91	74.52	74.08	75.53	83.81	33.71	65.78	70.55	meta-llama/Llama-3.1-70B-Instruct
⭕	CohereForAI/c4ai-command-r-08-2024	67.84	79.76	76.96	67	82.37	35.81	65.12	32.3	CohereForAI/c4ai-command-r-08-2024
🟢	meta-llama/Llama-3.2-90B-Vision	66.54	74.29	67.25	75.37	83.81	33.72	64.78	88.59	meta-llama/Llama-3.2-90B-Vision
🔶	618AI/dictalm2-it-qa-fine-tune	66.05	75.95	81.76	61.53	71.94	35.37	69.77	7.25	618AI/dictalm2-it-qa-fine-tune
🟢	mistralai/Mistral-Small-3.1-24B-Base-2503	66.03	81.19	78.01	66.73	80.94	30.81	58.47	24.01	mistralai/Mistral-Small-3.1-24B-Base-2503
⭕	mistralai/Mistral-Small-24B-Instruct-2501	65.86	83.1	75.79	71.67	79.5	29.95	55.15	23.57	mistralai/Mistral-Small-24B-Instruct-2501
🟢	nvidia/Nemotron-H-56B-Base-8K	65.11	83.57	78.4	69.77	77.34	29.1	52.49	56.3	nvidia/Nemotron-H-56B-Base-8K
🟢	google/gemma-3-12b-pt	65.02	85.71	77.35	59.6	74.82	34.47	58.14	12.19	google/gemma-3-12b-pt
🟢	mistralai/Mistral-Small-24B-Base-2501	64.82	83.1	78.21	64.8	80.22	30.74	51.83	23.57	mistralai/Mistral-Small-24B-Base-2501
⭕	CohereForAI/aya-23-35B	64.77	87.86	75.56	62.33	80.58	33.15	49.17	34.98	CohereForAI/aya-23-35B
🟦	Qwen/QwQ-32B-Preview	64.72	87.38	75.71	73.87	77.34	29.49	44.52	32.76	Qwen/QwQ-32B-Preview
⭕	nvidia/Llama-3_3-Nemotron-Super-49B-v1_5	64.19	80.71	72.9	74.4	76.62	29.65	50.83	49.9	nvidia/Llama-3_3-Nemotron-Super-49B-v1_5
🟢	dicta-il/dictalm2.0	63.89	79.05	75.69	59.67	66.91	35.57	66.45	7.25	dicta-il/dictalm2.0
⭕	rombodawg/Rombos-LLM-V2.5-Qwen-32b	63.42	90.24	67.76	77.27	76.26	28.11	40.86	32.76	rombodawg/Rombos-LLM-V2.5-Qwen-32b
⭕	dicta-il/dictalm2.0-instruct	62.94	73.57	76.9	56.3	69.42	35.3	66.11	7.25	dicta-il/dictalm2.0-instruct
⭕	ai21labs/AI21-Jamba-Mini-1.6	62.87	72.62	75.38	61.93	72.66	31.86	62.79	51.57	ai21labs/AI21-Jamba-Mini-1.6
⭕	Qwen/Qwen3-30B-A3B-Thinking-2507	62.67	89.05	75.03	72.07	70.14	28.52	41.2	30.53	Qwen/Qwen3-30B-A3B-Thinking-2507
⭕	ai21labs/AI21-Jamba-1.5-Large	62.62	83.1	65.49	60.7	81.65	12.36	72.43	398.56	ai21labs/AI21-Jamba-1.5-Large
⭕	Qwen/Qwen3-30B-A3B-Instruct-2507	62.61	90.48	75.02	71.37	69.42	27.83	41.53	30.53	Qwen/Qwen3-30B-A3B-Instruct-2507
🟢	Qwen/Qwen3-30B-A3B-Base	62.35	93.33	75.96	64.63	68.71	29.26	42.19	30.53	Qwen/Qwen3-30B-A3B-Base
🟢	Qwen/Qwen3-14B-Base	62.27	84.76	76.29	70.2	73.74	30.12	38.54	14.77	Qwen/Qwen3-14B-Base
🟦	google/gemma-2-9b-it	62.07	87.62	73.53	70.33	72.66	30.05	38.21	9.24	google/gemma-2-9b-it
🟢	Qwen/Qwen2.5-14B	62.01	90	74.39	71.73	68.35	28.71	38.87	14.77	Qwen/Qwen2.5-14B
🟦	UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3	61.91	88.57	71.89	70.7	71.22	29.85	39.2	9.24	UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
🟦	deepseek-ai/DeepSeek-R1-Distill-Qwen-32B	61.83	85.71	73.82	74	70.86	27.69	38.87	32.76	deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
🔶	deepseek-ai/DeepSeek-R1-Distill-Qwen-32B	61.76	85.95	73.83	73.7	70.5	27.71	38.87	32.76	deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
⭕	CohereForAI/c4ai-command-r-v01	61.69	73.57	73.15	59.07	75.54	30.99	57.81	34.98	CohereForAI/c4ai-command-r-v01
🟢	microsoft/phi-4	61.53	76.9	74.74	69.63	75.54	29.51	42.86	14.66	microsoft/phi-4
⭕	Qwen/Qwen3-32B	61.52	88.57	71.18	71.73	67.99	29.13	40.53	32.76	Qwen/Qwen3-32B
⭕	ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew	60.78	67.38	73.59	59.97	65.11	33.51	65.12	7.25	ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew
⭕	CohereLabs/c4ai-command-r7b-arabic-02-2025	60.63	83.57	74.62	61.63	71.94	32.15	39.87	8.03	CohereLabs/c4ai-command-r7b-arabic-02-2025
🔶	SicariusSicariiStuff/Impish_Nemo_12B	60.6	73.1	73.48	63.7	78.42	28.39	46.51	0	SicariusSicariiStuff/Impish_Nemo_12B
⭕	SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned	60.43	86.43	77.76	64.1	70.14	30.9	33.22	7.24	SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned
⭕	Qwen/Qwen3-14B	60.28	83.57	75.13	71.67	66.55	27.87	36.88	14.77	Qwen/Qwen3-14B
🟢	google/gemma-2-9b	59.9	73.57	75.71	65.23	70.86	31.49	42.52	9.24	google/gemma-2-9b
🔶	SeaLLMs/SeaLLM-7B-v2.5	59.72	79.05	76.31	67.03	70.86	27.17	37.87	8.54	SeaLLMs/SeaLLM-7B-v2.5
⭕	mistralai/Mistral-Nemo-Instruct-2407	59.21	72.38	73.49	65.77	72.66	27.41	43.52	12.25	mistralai/Mistral-Nemo-Instruct-2407
🟢	Qwen/Qwen2.5-32B	59.2	90	75.05	74.07	42.09	30.81	43.19	32.76	Qwen/Qwen2.5-32B
⭕	CohereForAI/aya-expanse-8b	59.1	77.14	73.02	63.4	71.58	31.24	38.21	8.03	CohereForAI/aya-expanse-8b
🟢	Qwen/Qwen3-8B-Base	58.8	75.71	75.59	70.4	67.27	27.64	36.21	8.19	Qwen/Qwen3-8B-Base
🔶	ronigold/dictalm2.0-instruct-fine-tuned	58.6	55.74	83.31	49.17	64.75	33.95	64.67	7.25	ronigold/dictalm2.0-instruct-fine-tuned
⭕	SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP	58.47	78.1	79.66	70.3	61.87	27.7	33.22	7.24	SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP
🔶	SicariusSicariiStuff/Zion_Alpha	58.44	84.05	72.6	65.67	65.83	27.93	34.55	7.24	SicariusSicariiStuff/Zion_Alpha
🔶	deepseek-ai/DeepSeek-R1-Distill-Qwen-14B	57.96	85.48	73.53	69.2	63.67	22.99	32.89	14.77	deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
🟢	Qwen/Qwen2.5-7B	57.35	81.9	73.84	66.73	62.59	25.5	33.55	7.62	Qwen/Qwen2.5-7B
⭕	Qwen/Qwen3-8B	56.99	80	74.69	68.3	63.31	25.73	29.9	8.19	Qwen/Qwen3-8B
🟢	mistralai/Mistral-Nemo-Base-2407	56.83	65.95	75.82	57.47	71.22	28.99	41.53	12.25	mistralai/Mistral-Nemo-Base-2407
⭕	ai21labs/AI21-Jamba-1.5-Mini	56.73	69.52	67.51	50.5	73.02	22	57.81	51.57	ai21labs/AI21-Jamba-1.5-Mini
🟦	mlabonne/NeuralDaredevil-8B-abliterated	55.42	78.81	67.52	57.57	66.55	26.2	35.88	8.03	mlabonne/NeuralDaredevil-8B-abliterated
⭕	CohereForAI/aya-23-8B	55.3	65.71	73.84	59.23	66.55	30.26	36.21	8.03	CohereForAI/aya-23-8B
🔶	NousResearch/Hermes-3-Llama-3.1-8B	55.28	64.52	69.88	66.77	67.63	26.68	36.21	8.03	NousResearch/Hermes-3-Llama-3.1-8B
🟦	meta-llama/Meta-Llama-3.1-8B-Instruct	55.13	73.81	72.27	57.87	69.42	25.86	31.56	8.03	meta-llama/Meta-Llama-3.1-8B-Instruct
🟢	nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base	55.02	71.43	73.82	64.47	64.03	21.51	34.88	12.3	nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base
🟢	google/gemma-2-27b	55.01	63.81	65.15	58.73	75.54	17.99	48.84	27.23	google/gemma-2-27b
🟢	meta-llama/Llama-3.2-11B-Vision	54.94	72.14	72.25	58.73	68.35	25.96	32.23	10.64	meta-llama/Llama-3.2-11B-Vision
🔶	mistralai/Codestral-22B-v0.1	54.82	74.76	70.39	64.8	64.75	24.65	29.57	22.25	mistralai/Codestral-22B-v0.1
🟦	vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B	54.63	79.29	65.02	63.73	62.95	25.56	31.23	8.03	vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B
⭕	mistralai/Ministral-8B-Instruct-2410	54.49	56.43	76.95	63.1	67.99	25.25	37.21	8.02	mistralai/Ministral-8B-Instruct-2410
🟢	Qwen/Qwen1.5-32B	54.27	67.86	71.61	66.93	61.15	26.52	31.56	32.51	Qwen/Qwen1.5-32B
🟢	Qwen/Qwen3-4B-Base	54.23	72.86	74.58	60.17	61.51	24.71	31.56	4.02	Qwen/Qwen3-4B-Base
🟢	meta-llama/Meta-Llama-3.1-8B	53.73	64.05	74.07	56.03	66.19	27.52	34.55	8.03	meta-llama/Meta-Llama-3.1-8B
🟢	yam-peleg/Hebrew-Mistral-7B	53.42	59.52	73.52	48.57	64.75	28.98	45.18	7.5	yam-peleg/Hebrew-Mistral-7B
⭕	mistralai/Mistral-Small-Instruct-2409	53.34	63.33	73.64	62.57	64.75	22.86	32.89	22.25	mistralai/Mistral-Small-Instruct-2409
🟦	MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct	53.33	77.86	70.72	52.67	67.63	25.54	25.58	8.03	MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct
🟢	google/gemma-7b	53.1	59.76	75.86	62.73	62.59	27.93	29.73	8.54	google/gemma-7b
🟦	Danielbrdz/Barcenas-Llama3-8b-ORPO	52.95	74.52	68.84	51.4	64.75	24.61	33.55	8.03	Danielbrdz/Barcenas-Llama3-8b-ORPO
⭕	yam-peleg/Hebrew-Gemma-11B-Instruct	52.55	55.48	72.89	69.7	59.71	27.32	30.23	10.48	yam-peleg/Hebrew-Gemma-11B-Instruct
⭕	Qwen/Qwen3-4B	52.36	64.05	73.24	69.07	58.99	21.25	27.57	4.02	Qwen/Qwen3-4B

Hebrew LLM Leaderboard