Fine-tuning strips propaganda from a Chinese model

How propaganda was measured

Nous created a test of 803 prompts across seven categories, including freedom of speech, state ideology, education, privacy and surveillance, democracy, and the environment. For each prompt, the team selected two to four facts from a dated source: a law, news article or research paper.

An answer was counted as propaganda if it repeated Beijing’s official position instead of the facts, endorsed that position or presented it as an equally valid point of view. A judge model checked whether the facts were mentioned, omitted or distorted; it was not told which model had produced the answer. The judge was xAI’s Grok 4.7, which Nous describes as leaning politically to the right. The judge had to cite the specific wording behind its assessment.

The original Qwen3.8-27B scored 2.32 out of 5 for factual accuracy, and 22% of its answers were classified as propaganda. The figure was below 1% for the Western models Mistral Large 4 and Gemma 4 31B. It was 15% for GLM-5.3 and 16% for DeepSeek V4.1 Flash.

Fine-tuning and results

For knowledge transfer, Nous chose Inkling from Thinking Machines Lab. The company said an open model would not violate the terms of use that prohibit distillation. Nvidia’s Nemotron 3 Ultra was also on the shortlist, but Nous reported signs of anomalies in answers on China-related topics. The team suspects the bias may have carried over from the training data of Chinese open models; it said it had seen similar contamination in its own projects. Thinking Machines used several models, including the Chinese Kimi K2.5, to generate Inkling’s synthetic training data.

Qwen3.8-27B was supervised fine-tuned on about 53 million tokens over two epochs. The team did not use reinforcement learning or preference optimization. Before training, it removed overlaps between the training data and test prompts.

After fine-tuning, Arete-27B scored 2.98 for factual accuracy, while the share of answers classified as propaganda fell to 0.9%. The change was especially pronounced in the freedom-of-speech category: the factual score rose from 0.89 to 2.44, and the share of propaganda answers fell from 60% to 1%. In reasoning mode, the model scored 3.07, but that result is based on only two of five runs so far.

The training data is spread nearly evenly across the seven value categories, with most examples running under 1,000 tokens. | Image: Nous Research

Source: the-decoder.com

According to Nous, the model’s general capabilities were largely unaffected. On the company’s internal Hermes Bench, Arete-27B scored 65.60%, compared with 65.77% for the original model. Its Terminal-Bench 2.1 score rose from 67% to 70%, but Nous does not consider this a confirmed improvement because the testing conditions differed.

Arete-27B's propaganda rate is on par with Western models, while GLM 5.3, DeepSeek V4.1 Flash, and the base model Qwen3.8-27B score much higher. | Image: Nous Research

Source: the-decoder.com

The company concluded that the layer of political bias in a model aligned with Chinese positions is shallow. In Nous’s assessment, it is introduced late in training, so carefully selected fine-tuning data is enough to remove it without affecting the model’s other capabilities.

From neutrality to Western values

With Arete, Nous is moving away from the approach it took with Hermes 3. In 2024, that model was deliberately made neutral: it was meant to adapt to the worldview specified in the system prompt. Arete, by contrast, is positioned as a model that adheres to “Western values” rather than Beijing’s party line. Nous is an American startup also known for the open-source Hermes AI agent.

Earlier research has also found propaganda distortions in Chinese models. In 2025, NewsGuard reported that five leading models, including Qwen, repeated pro-China disinformation or failed to correct it in 60% of cases.

CrowdStrike found that DeepSeek writes less secure code when prompts mention Falun Gong, Tibet or Taiwan. In a study by Aleph Alpha involving 967 politically sensitive questions, between 17% and 41% of answers from models by Alibaba, DeepSeek and Moonshot AI were balanced.

A later study by ValsAI found that Chinese models behave differently: GLM-5.3 Flash directly acknowledged human rights violations, while Kimi K3 denied the existence of internment camps. The language of the prompt and the response format had a substantial effect on results; Western models sometimes adopted Beijing’s official positions as well. The study’s authors also noted that targeted fine-tuning helped change answers.

In October, Nous Research raised $90 million in a Series B round at a $1.5 billion valuation. Robot Ventures led the round, with Nvidia and Samsung among the participants. The company is using the funds to enter the enterprise market with “Hermes for Businesses.” Arete-27B’s weights, test set and training data are available on Hugging Face under the Apache 2.0 license.

Dataist's take

The results suggest that some political bias in open models can be reduced without a noticeable drop in general test scores. However, the metrics depend on the prompts chosen, the evaluation criteria and the judge model, and Arete’s result in reasoning mode is still based on just two of five runs.