Introduction
The KCL team is pleased to announce that KCL v0.6.0 is now available! This release has brought three key updates to everyone: Language, Tools, and Integrations.
- Use KCL language, tools and IDE extensions with more complete features and fewer errors to improve code writing experience and efficiency.
- Use KPM, OCI Registry and other tools to directly use and share your cloud native domain models, reducing learning and hands-on costs.
- Use cloud-native integration extensions such as Helmfile KCL plugin and KCL Operator to simultaneously support in-place mutation and validation of Kubernetes resources on both the client and runtime, avoiding hardcoded configurations.
You can visit the KCL release page or the KCL website to get KCL binary download link and more detailed release information.
KCL is an open-source, constraint-based record and functional language. KCL improves the writing of numerous complex configurations, such as cloud-native scenarios, through its mature programming language technology and practice. It is dedicated to building better modularity, scalability, and stability around configurations, simpler logic writing, faster automation, and great built-in or API-driven integrations.
This blog will introduce the content of KCL v0.6.0 and recent developments in the KCL community to readers.
Language
🔧 Type system enhancement
Support automatic type inference for KCL configuration block attributes. Prior to version 0.6.0 of KCL, the key1 and key2 attributes in the code snippet below would be inferred as type str | int. With the updated version, we have further improved the precision of type inference for configuration attributes, so key1 and key2 will have more specific and precise corresponding types.
config = {
key1 = "value1"
key2 = 2
}
key1 = config.key1 # The type of key1 is str
key2 = config.key2 # The type of key2 is int
In addition, we have optimized error messages for schema semantic checking, union type checking, and type checking errors in system library functions.
For more information, please refer to here
🏄 API Update
KCL Schema model parsing: The GetSchemaType API is used to retrieve KCL package-related information and default values for schema attributes.
🐞 Bug Fixes
Fix schema required/optional attribute check in KCL
In previous versions of KCL, the required/optional attribute check for KCL would miss nested schema attribute checks. In version KCL v0.6.0, we have fixed such similar issues.
schema S:
a: int
b: str
schema L:
# In previous versions, the required attribute check for attributes 'a' and 'b' of S in [S] and {str:S} would be missed.
# This issue has been fixed in KCL v0.6.0 and later versions.
ss?: [S]
sss?: {str:S}
l = L {
ss = [S {b = "b"}]
}
For more information, please see here
IDE & Toolchain Updates
IDE Updates
Features
- Performance improvement for all IDE features
- Support variable and schema attribute completion for KCL packages
- Support KCL schema attribute document attribute hover
- Support for quick fix of useless import statements
- Support right-click formatting of files and code fragments in VS Code.
- Support hover for built-in functions and function information in system libraries
IDE Extension Updates
We have integrated the KCL language server LSP into NeoVim and Idea, enabling the completion, navigation, and hover features supported by VS Code IDE in NeoVim and IntelliJ IDEA.
- NeoVim KCL Extension
- IntelliJ Extension
For more information on downloading, installation, and features of the IDE plugins, please refer to:
- https://kcl-lang.io/docs/user_docs/getting-started/install#neovim
- https://kcl-lang.io/docs/user_docs/getting-started/install#intellij-idea
KCL Formatting Tool Updates
Support formatting of configuration blocks with incorrect indentation
- Before formatting
config = {
a ={
x = 1
y =2
}
b = {
x = 1
y = 2
}
}
- After formatting
config = {
a = {
x = 1
y = 2
}
b = {
x = 1
y = 2
}
}
KCL Documentation Tool Updates
- Support for exporting Markdown documents
- Support for exporting document index pages
- Support for exporting documents with custom style templates
- Support for HTML escaping in exported documents
- Enhanced document generation to parse and render example code snippets in document comments
- By tracking model updates in Github workflow and regenerating the documentation, automatic synchronization of the documentation can be achieved. Please refer to here for more details.
Generate model document from kpm package
- Create a kpm package and add documentation comments (using docstring) to the
Servicemodel. The documentation can include explanations, example code, and usage instructions to help other developers quickly get started and use it correctly.
➜ kpm init demo
➜ cat > demo/server.k << EOF
schema Service:
"""
Service is a kind of workload profile that describes how to run your application code. This
is typically used for long-running web applications that should "never" go down, and handle
short-lived latency-sensitive web requests, or events.
Attributes
----------
workloadType : str = "Deployment" | "StatefulSet", default is Deployment, required.
workloadType represents the type of workload used by this Service. Currently, it supports several
types, including Deployment and StatefulSet.
image : str, default is Undefined, required.
Image refers to the Docker image name to run for this container.
More info: https://kubernetes.io/docs/concepts/containers/images
replicas : int, default is 2, required.
Number of container replicas based on this configuration that should be ran.
Examples
--------
# Instantiate a long-running service and its image is "nginx:v1"
svc = Service {
workloadType: "Deployment"
image: "nginx:v1"
replica: 2
}
"""
workloadType: "Deployment" | "StatefulSet"
image: str
replica: int
EOF
- Generate the package documentation in Markdown format
The following command will output the demo package documentation to the doc/ directory in the current working directory:
kcl-go doc generate --file-path demo
For more usage details, please use
kcl-go doc generate -hto refer to the help information.
Automatic synchronization of documents through CI pipelines
Implement automatic documentation synchronization through a pipeline By tracking model updates in a Github workflow and regenerating the documentation, automatic synchronization of the documentation can be achieved. You can refer to the approach in the Kusionstack/catalog repo to generate the documentation and automatically submit change PRs to the documentation repository.
By tracking model updates in Github workflow and regenerating documents, automatic document synchronization can be achieved. Can refer to the approach in the Kusionstack/catalog repo is to generate documents and automatically submit change PRs to the document repository.
KCL Import Tool Updates
The KCL Import Tool now adds support for converting Terraform Provider Schema to KCL Schema based on Protobuf, JsonSchema OpenAPI models, and Go Structures, such as the following Terraform Provider Json (obtained through the command terraform providers schema -json > provider.json , For more details, please refer to https://developer.hashicorp.com/terraform/cli/commands/providers/schema)
{
"format_version": "0.2",
"provider_schemas": {
"registry.terraform.io/aliyun/alicloud": {
"provider": {
"version": 0,
"block": {
"attributes": {},
"block_types": {},
"description_kind": "plain"
}
},
"resource_schemas": {
"alicloud_db_instance": {
"version": 0,
"block": {
"attributes": {
"db_instance_type": {
"type": "string",
"description_kind": "plain",
"computed": true
},
"engine": {
"type": "string",
"description_kind": "plain",
"required": true
},
"security_group_ids": {
"type": [
"set",
"string"
],
"description_kind": "plain",
"optional": true,
"computed": true
},
"security_ips": {
"type": [
"set",
"string"
],
"description_kind": "plain",
"optional": true,
"computed": true
},
"tags": {
"type": [
"map",
"string"
],
"description_kind": "plain",
"optional": true
}
},
"block_types": {},
"description_kind": "plain"
}
},
"alicloud_config_rule": {
"version": 0,
"block": {
"attributes": {
"compliance": {
"type": [
"list",
[
"object",
{
"compliance_type": "string",
"count": "number"
}
]
],
"description_kind": "plain",
"computed": true
},
"resource_types_scope": {
"type": [
"list",
"string"
],
"description_kind": "plain",
"optional": true,
"computed": true
}
}
}
}
},
"data_source_schemas": {}
}
}
}
Then the tool can output the following KCL code
"""
This file was generated by the KCL auto-gen tool. DO NOT EDIT.
Editing this file might prove futile when you re-run the KCL auto-gen generate command.
"""
schema AlicloudConfigRule:
"""
AlicloudConfigRule
Attributes
----------
compliance: [ComplianceObject], optional
resource_types_scope: [str], optional
"""
compliance?: [ComplianceObject]
resource_types_scope?: [str]
schema ComplianceObject:
"""
ComplianceObject
Attributes
----------
compliance_type: str, optional
count: int, optional
"""
compliance_type?: str
count?: int
schema AlicloudDbInstance:
"""
AlicloudDbInstance
Attributes
----------
db_instance_type: str, optional
engine: str, required
security_group_ids: [str], optional
security_ips: [str], optional
tags: {str:str}, optional
"""
db_instance_type?: str
engine: str
security_group_ids?: [str]
security_ips?: [str]
tags?: {str:str}
check:
isunique(security_group_ids)
isunique(security_ips)
Package Manage Tool Updates
kpm pull supports pulling packages by package name
kpm supports pulling the corresponding package by using the kpm pull <package_name>:<package_version> command.
Taking the k8s package as an example, you can directly download the package to your local machine using the following commands:
kpm pull k8s
or
kpm pull k8s:1.27
The package downloaded with kpm pull will be saved in the directory <execution_directory>/<oci_registry>/<package_name>. For example, if you use the default kpm registry and run the kpm pull k8s command, you can find the downloaded content in the directory <execution_directory>/ghcr.io/kcl-lang/k8s.
$ tree ghcr.io/kcl-lang/k8s -L 1
ghcr.io/kcl-lang/k8s
├── api
├── apiextensions_apiserver
├── apimachinery
├── kcl.mod
├── kcl.mod.lock
├── kube_aggregator
└── vendor
6 directories, 2 files
kpm supports adding local paths as dependencies
"Different projects have different KCL packages, and there are dependencies between them. However, they are stored in different directories. I hope that these packages stored in different directories can be managed together, rather than having to put them together for them to compile." If you also have this need, you can try this feature. The kpm add command currently supports adding local paths as dependencies to a KCL package. You just need to run the command kpm add <local_package_path>, and your local package will be added as a third-party library dependency to the current package.
kpm pull k8s
After completion, you can find the downloaded k8s package in the directory "the_directory_where_you_executed_the_command/ghcr.io/kcl-lang/k8s".
Create a new KCL package using the kpm init mynginx command.
kpm init mynginx
Then, navigate into this package.
cd mynginx
Inside this package, you can use the kpm add command to add the k8s package you downloaded locally as a third-party library dependency to mynginx.
kpm add ../ghcr.io/kcl-lang/k8s/
Next, add the following content to main.k.
import k8s.api.core.v1 as k8core
k8core.Pod {
metadata.name = "web-app"
spec.containers = [{
name = "main-container"
image = "nginx"
ports = [{containerPort: 80}]
}]
}
Normal compilation can be performed through the kpm run command.
kpm run
kpm adds checking for existing package tags
We have added a check for duplicate tags in the kpm push command. In order to avoid situations where packages with the same tag have different content, we have added restrictions on the push function in the kpm. If the version of the kcl package you push already exists, you will not be able to push the current kcl package. You will receive the following information:
kpm: package 'my_package' will be pushed.
kpm: package version '0.1.0' already exists
Modifying the content of a package that has already been pushed to the registry without changing the tag carries a high risk, as the package may already be in use by others. Therefore, if you need to push your package, we recommend:
- Change your tag and follow semantic versioning conventions.
- If you must modify the content of a package without changing the tag, you will need to delete the existing tag from the registry.
Integrations
Helmfile KCL Plugin
Helmfile is a declarative specification and tool for deploying Helm Charts. With the Helmfile KCL plugin, you can:
- Edit or verify Helm Chart through non-invasive hook methods, separating the data and logic parts of Kubernetes configuration management
- Modify resource labels/annotations, inject sidecar container configuration
- Use KCL schema to validate resources
- Define your own abstract application models
- Maintain multiple environment and tenant configurations elegantly, rather than simply copying and pasting.
Here is a detailed explanation using a simple example. With the Helmfile KCL plugin, you do not need to install any components related to KCL. You only need the latest version of the Helmfile tool on your local device.
We can write a helmfile.yaml file as follows:
repositories:
- name: prometheus-community
url: https://prometheus-community.github.io/helm-charts
releases:
- name: prom-norbac-ubuntu
namespace: prometheus
chart: prometheus-community/prometheus
set:
- name: rbac.create
value: false
transformers:
# Use KCL Plugin to mutate or validate Kubernetes manifests.
- apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
name: "set-annotation"
annotations:
config.kubernetes.io/function: |
container:
image: docker.io/kcllang/kustomize-kcl:v0.2.0
spec:
source: |
[resource | {if resource.kind == "Deployment": metadata.annotations: {"managed-by" = "helmfile-kcl"}} for resource in option("resource_list").items]
In the above file, we referenced the Prometheus Helm Chart and injected the managed-by="helmfile-kcl" label into all deployment resources of Prometheus with just one line of KCL code. The following command can be used to deploy the above configuration to the Kubernetes cluster:
helmfile apply
For more use cases, please refer to https://github.com/kcl-lang/krm-kcl
KCL Operator
KCL Operator provides cluster integration, allowing you to use Access Webhook to generate, mutate, or validate resources based on KCL configuration when apply resources to the cluster. Webhook will capture creation, application, and editing operations, and execute KCLRun on the configuration associated with each operation, and the KCL programming language can be used to
- Add labels or annotations based on a condition.
- Inject a sidecar container in all KRM resources that contain a
PodTemplate. - Validating all KRM resources using KCL Schema, such as constraints on starting containers only in a root mode.
- Generating KRM resources using an abstract model or combining and using different KRM APIs.
Here is a simple resource annotation mutation example to introduce the usage of the KCL operator.
0. Prerequisites
Prepare a Kubernetes cluster like k3d the kubectl tool.
1. Install KCL Operator
kubectl apply -f https://raw.githubusercontent.com/kcl-lang/kcl-operator/main/config/all.yaml
Use the following command to observe and wait for the pod status to be Running.
kubectl get po
2. Deploy KCL Annotation Setting Model
kubectl apply -f- << EOF
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
name: set-annotation
spec:
# Set dynamic parameters required for the annotation modification model, here we can add the labels we want to modify/add
params:
annotations:
managed-by: kcl-operator
# Reference the annotation modification model on OCI
source: oci://ghcr.io/kcl-lang/set-annotation
EOF
3. Deploy a Pod to Verify the Model Result
Execute the following command to deploy a Pod resource:
kubectl apply -f- << EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
annotations:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
EOF
kubectl get po nginx -o yaml | grep kcl-operator
We can see the following output:
managed-by: kcl-operator
We can see that the Nginx Pod automatically added the annotation managed-by=kcl-operator.
In addition, besides referencing an existing model for the source field of the KCLRun resource, we can directly set KCL code for the source field to achieve the same effect. For example:
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
name: set-annotation
spec:
params:
annotations:
managed-by: kcl-operator
# Resource modification can be achieved with just one line of KCL code
source: |
items = [item | {metadata.annotations: option("params").annotations} for item in option("items")]
We have provided more than 30 built-in models, and you can find more code examples in the following link: https://github.com/kcl-lang/krm-kcl/tree/main/examples
For example
- Use the
web-servicemodel to directly instantiate the Kubernetes resources required for a web application - Add annotations to existing k8s resources using the
set-annotationmodel - Use the
https-onlymodel to verify that yourIngressconfiguration can only be set to https, otherwise an error will be reported.
Vault Integration
In just three steps, we can use Vault to store and manage sensitive information and use it in KCL.
Firstly, we install and use Vault to store foo and bar information.
vault kv put secret/foo foo=foo
vault kv put secret/bar bar=bar
Then write the following KCL code (main.k)
apiVersion = "apps/v1"
kind = "Deployment"
metadata = {
name = "nginx"
labels.app = "nginx"
annotations: {
"secret-store": "vault"
# Valid format:
# "ref+vault://PATH/TO/KV_BACKEND#/KEY"
"foo": "ref+vault://secret/foo#/foo"
"bar": "ref+vault://secret/bar#/bar"
}
}
spec = {
replicas = 3
selector.matchLabels = metadata.labels
template.metadata.labels = metadata.labels
template.spec.containers = [
{
name = metadata.name
image = "${metadata.name}:1.14.2"
ports = [{ containerPort = 80 }]
}
]
}
Finally, the decrypted configuration can be obtained through the vals command-line tool
kcl main.k | vals eval -f -
For more details and use cases, please refer to here
GitLab CI Integration
Using KCL, we can not only use Github Action as CI for application publishing through GitOps, but also provide GitLab CI integration in this version. Please refer to: https://kcl-lang.io/docs/user_docs/guides/ci-integration/gitlab-ci
Other Updates and Bug Fixes
See here for more updates and bug fixes.
Documents
The versioning semantic option is added to the KCL website. Currently, v0.4.x, v0.5.x and v0.6.0 versions are supported.
Community
- Thank @jakezhu9 for his contribution to converting KCL Import tools, including Terraform Provider Schema, JsonSchema, JSON, YAML, and other configuration formats/data to KCL schema/configurations 🙌
- Thank @xxmao123 for her contribution to connecting KCL LSP language server to the Idea IDE extension 🙌
- Thank @starkers for his contribution to the KCL NeoVim extension 🙌
- Thank @starkers for adding KCL installation support to mason.nvim registry 🙌
- Thank @Ekko for his contribution to the integration of KCL cloud native tools and KCL Operator 🙌
- Thank @prahalaramji for the upgrade, update, and contribution to the KCL Homebrew installation script 🙌
- Thank @yyxhero for providing assistance and support in the Helmfile KCL plugin support 🙌
- Thank @nkabir, @mihaigalos, @prahalaramji, @yamin-oanda, @dhhopen, @magick93, @MirKml, @kolloch, @steeling, and others for their valuable feedback and discussion during the past two months of using KCL. 🙌
Additional Resources
For more information, see KCL FAQ.
Resources
Thank all KCL users for their valuable feedback and suggestions during this version release. For more resources, please refer to:
See the community for ways to join us. 👏👏👏
