StableLM: 稳定性AI的语言模型

Stability AI推出的StableLM系列包括多个不同规模和功能的模型，旨在不断优化和扩展语言处理的边界。每个模型都具备独特的架构和训练策略，以适应不同的应用需求和性能目标。

index_open_StableLM_1

模型架构与训练详情

StableLM-3B-4E1T模型采用解码器唯一的变换器架构，与LLaMA模型类似，但进行了特定的调整以优化性能和处理能力。例如，该模型使用了旋转位置嵌入和LayerNorm标准化，以及GPT-NeoX分词器。

此外，StableLM-Alpha v2模型通过引入SwiGLU激活函数和更高质量的数据源，显著提高了模型性能。这些模型的上下文长度为4096令牌，支持复杂和细致的语言理解任务。

训练数据与效果评估

StableLM模型使用了多种开源大规模数据集，如Falcon RefinedWeb抽取和RedPajama-Data，通过这些高质量数据源确保了模型的综合性和适应性。模型的性能通过多种零样本评估任务来验证，确保其在多个标准化测试中的表现与竞争模型相当甚至更优。

如何开始使用

StableLM模型托管在Hugging Face平台，您可以通过简单的代码示例来启动并与StableLM-Tuned-Alpha模型交互，体验其独特的语言生成能力。

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList

tokenizer = AutoTokenizer.from_pretrained("stabilityai/stablelm-tuned-alpha-7b")
model = AutoModelForCausalLM.from_pretrained("stabilityai/stablelm-tuned-alpha-7b")
model.half().cuda()

class StopOnTokens(StoppingCriteria):
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
        stop_ids = set([50278, 50279, 50277, 1, 0])
        return input_ids[0][-1] in stop_ids

system_prompt = """<|SYSTEM|># StableLM Tuned (Alpha version)
- StableLM is a helpful and harmless open-source AI language model developed by StabilityAI.
- StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes.
- StableLM will refuse to participate in anything that could harm a human.
"""

prompt = f"{system_prompt}<|USER|>What's your mood today?<|ASSISTANT|>"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
tokens = model.generate(
  **inputs,
  max_new_tokens=64,
  temperature=0.7,
  do_sample=True,
  stopping_criteria=StoppingCriteriaList([StopOnTokens()])
)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList

tokenizer = AutoTokenizer.from_pretrained("stabilityai/stablelm-tuned-alpha-7b")
model = AutoModelForCausalLM.from_pretrained("stabilityai/stablelm-tuned-alpha-7b")
model.half().cuda()

class StopOnTokens(StoppingCriteria):
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
        stop_ids = set([50278, 50279, 50277, 1, 0])
        return input_ids[0][-1] in stop_ids

system_prompt = """<|SYSTEM|># StableLM Tuned (Alpha version)
- StableLM is a helpful and harmless open-source AI language model developed by StabilityAI.
- StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes.
- StableLM will refuse to participate in anything that could harm a human.
"""

prompt = f"{system_prompt}<|USER|>What's your mood today?<|ASSISTANT|>"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
tokens = model.generate(
  **inputs,
  max_new_tokens=64,
  temperature=0.7,
  do_sample=True,
  stopping_criteria=StoppingCriteriaList([StopOnTokens()])
)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))

通过实际操作StableLM模型，我深刻感受到了其在处理复杂语言任务时的强大能力和灵活性。无论是进行日常对话还是生成技术内容，StableLM都表现出色。对于开发者和研究人员来说，这不仅是一个工具，更是一个探索人工智能潜能的平台。