They clearly used ChatGPT as the basis for their reinforcement learning with human feedback question/answer pairs. Meaning it's going to have all the same censorship as ChatGPT does. There are models that have been scrubbed of any "I'm sorry as an AI language model I can't do/answer/etc" including ones related to "mean" output (Vicuna uncensored for instance, some Alpaca and other Llama variants).