KCL v0.6.0 Release Blog

Introduction

The KCL team is pleased to announce that KCL v0.6.0 is now available! This release has brought three key updates to everyone: Language, Tools, and Integrations.

• Use KCL language, tools and IDE extensions with more complete features and fewer errors to improve code writing experience and efficiency.
• Use KPM, OCI Registry and other tools to directly use and share your cloud native domain models, reducing learning and hands-on costs.
• Use cloud-native integration extensions such as Helmfile KCL plugin and KCL Operator to simultaneously support in-place mutation and validation of Kubernetes resources on both the client and runtime, avoiding hardcoded configurations.

You can visit the KCL release page or the KCL website to get KCL binary download link and more detailed release information.

KCL is an open-source, constraint-based record and functional language. KCL improves the writing of numerous complex configurations, such as cloud-native scenarios, through its mature programming language technology and practice. It is dedicated to building better modularity, scalability, and stability around configurations, simpler logic writing, faster automation, and great built-in or API-driven integrations.

This blog will introduce the content of KCL v0.6.0 and recent developments in the KCL community to readers.

Language

🔧 Type system enhancement

Support automatic type inference for KCL configuration block attributes. Prior to version 0.6.0 of KCL, the key1 and key2 attributes in the code snippet below would be inferred as type str | int. With the updated version, we have further improved the precision of type inference for configuration attributes, so key1 and key2 will have more specific and precise corresponding types.

config = {
    key1 = "value1"
    key2 = 2
}
key1 = config.key1  # The type of key1 is str
key2 = config.key2  # The type of key2 is int

In addition, we have optimized error messages for schema semantic checking, union type checking, and type checking errors in system library functions.

For more information, please refer to here

🏄 API Update

KCL Schema model parsing: The GetSchemaType API is used to retrieve KCL package-related information and default values for schema attributes.

🐞 Bug Fixes

Fix schema required/optional attribute check in KCL

In previous versions of KCL, the required/optional attribute check for KCL would miss nested schema attribute checks. In version KCL v0.6.0, we have fixed such similar issues.

schema S:
    a: int
    b: str
schema L:
    # In previous versions, the required attribute check for attributes 'a' and 'b' of S in [S] and {str:S} would be missed.
    # This issue has been fixed in KCL v0.6.0 and later versions.
    ss?: [S]
    sss?: {str:S}
l = L {
    ss = [S {b = "b"}]
}

For more information, please see here

IDE & Toolchain Updates

IDE Updates

Features

• Performance improvement for all IDE features
• Support variable and schema attribute completion for KCL packages
• Support KCL schema attribute document attribute hover
• Support for quick fix of useless import statements

• Support right-click formatting of files and code fragments in VS Code.

• Support hover for built-in functions and function information in system libraries

IDE Extension Updates

We have integrated the KCL language server LSP into NeoVim and Idea, enabling the completion, navigation, and hover features supported by VS Code IDE in NeoVim and IntelliJ IDEA.

• NeoVim KCL Extension

• IntelliJ Extension

For more information on downloading, installation, and features of the IDE plugins, please refer to:

• https://kcl-lang.io/docs/user_docs/getting-started/install#neovim
• https://kcl-lang.io/docs/user_docs/getting-started/install#intellij-idea

KCL Formatting Tool Updates

Support formatting of configuration blocks with incorrect indentation

• Before formatting

config = {
a ={
x = 1
 y =2
}
b = {
 x = 1
 y = 2
}
}

• After formatting

config = {
    a = {
        x = 1
        y = 2
    }
    b = {
        x = 1
        y = 2
    }
}

KCL Documentation Tool Updates

• Support for exporting Markdown documents
• Support for exporting document index pages
• Support for exporting documents with custom style templates
• Support for HTML escaping in exported documents
• Enhanced document generation to parse and render example code snippets in document comments
• By tracking model updates in Github workflow and regenerating the documentation, automatic synchronization of the documentation can be achieved. Please refer to here for more details.

Generate model document from kpm package

1. Create a kpm package and add documentation comments (using docstring) to the Service model. The documentation can include explanations, example code, and usage instructions to help other developers quickly get started and use it correctly.

➜ kpm init demo

➜ cat > demo/server.k << EOF
schema Service:
    """
    Service is a kind of workload profile that describes how to run your application code. This
    is typically used for long-running web applications that should "never" go down, and handle
    short-lived latency-sensitive web requests, or events.
    
    Attributes
    ----------
    workloadType : str = "Deployment" | "StatefulSet", default is Deployment, required.
        workloadType represents the type of workload used by this Service. Currently, it supports several
        types, including Deployment and StatefulSet.
    image : str, default is Undefined, required.
        Image refers to the Docker image name to run for this container.
        More info: https://kubernetes.io/docs/concepts/containers/images
    replicas : int, default is 2, required.
        Number of container replicas based on this configuration that should be ran.
    
    Examples
    --------
    # Instantiate a long-running service and its image is "nginx:v1"

    svc = Service {
        workloadType: "Deployment"
        image: "nginx:v1"
        replica: 2
    }
    """
    workloadType: "Deployment" | "StatefulSet"
    image: str
    replica: int
EOF

2. Generate the package documentation in Markdown format

The following command will output the demo package documentation to the doc/ directory in the current working directory:

kcl-go doc generate --file-path demo

For more usage details, please use kcl-go doc generate -h to refer to the help information.

Automatic synchronization of documents through CI pipelines

Implement automatic documentation synchronization through a pipeline By tracking model updates in a Github workflow and regenerating the documentation, automatic synchronization of the documentation can be achieved. You can refer to the approach in the Kusionstack/catalog repo to generate the documentation and automatically submit change PRs to the documentation repository.

By tracking model updates in Github workflow and regenerating documents, automatic document synchronization can be achieved. Can refer to the approach in the Kusionstack/catalog repo is to generate documents and automatically submit change PRs to the document repository.

KCL Import Tool Updates

The KCL Import Tool now adds support for converting Terraform Provider Schema to KCL Schema based on Protobuf, JsonSchema OpenAPI models, and Go Structures, such as the following Terraform Provider Json (obtained through the command terraform providers schema -json > provider.json , For more details, please refer to https://developer.hashicorp.com/terraform/cli/commands/providers/schema)

{
    "format_version": "0.2",
    "provider_schemas": {
        "registry.terraform.io/aliyun/alicloud": {
            "provider": {
                "version": 0,
                "block": {
                    "attributes": {},
                    "block_types": {},
                    "description_kind": "plain"
                }
            },
            "resource_schemas": {
                "alicloud_db_instance": {
                    "version": 0,
                    "block": {
                        "attributes": {
                            "db_instance_type": {
                                "type": "string",
                                "description_kind": "plain",
                                "computed": true
                            },
                            "engine": {
                                "type": "string",
                                "description_kind": "plain",
                                "required": true
                            },
                            "security_group_ids": {
                                "type": [
                                    "set",
                                    "string"
                                ],
                                "description_kind": "plain",
                                "optional": true,
                                "computed": true
                            },
                            "security_ips": {
                                "type": [
                                    "set",
                                    "string"
                                ],
                                "description_kind": "plain",
                                "optional": true,
                                "computed": true
                            },
                            "tags": {
                                "type": [
                                    "map",
                                    "string"
                                ],
                                "description_kind": "plain",
                                "optional": true
                            }
                        },
                        "block_types": {},
                        "description_kind": "plain"
                    }
                },
                "alicloud_config_rule": {
                    "version": 0,
                    "block": {
                        "attributes": {
                            "compliance": {
                                "type": [
                                    "list",
                                    [
                                        "object",
                                        {
                                            "compliance_type": "string",
                                            "count": "number"
                                        }
                                    ]
                                ],
                                "description_kind": "plain",
                                "computed": true
                            },
                            "resource_types_scope": {
                                "type": [
                                    "list",
                                    "string"
                                ],
                                "description_kind": "plain",
                                "optional": true,
                                "computed": true
                            }
                        }
                    }
                }
            },
            "data_source_schemas": {}
        }
    }
}

Then the tool can output the following KCL code

"""
This file was generated by the KCL auto-gen tool. DO NOT EDIT.
Editing this file might prove futile when you re-run the KCL auto-gen generate command.
"""

schema AlicloudConfigRule:
    """
    AlicloudConfigRule

    Attributes
    ----------
    compliance: [ComplianceObject], optional
    resource_types_scope: [str], optional
    """

    compliance?: [ComplianceObject]
    resource_types_scope?: [str]

schema ComplianceObject:
    """
    ComplianceObject

    Attributes
    ----------
    compliance_type: str, optional
    count: int, optional
    """

    compliance_type?: str
    count?: int

schema AlicloudDbInstance:
    """
    AlicloudDbInstance

    Attributes
    ----------
    db_instance_type: str, optional
    engine: str, required
    security_group_ids: [str], optional
    security_ips: [str], optional
    tags: {str:str}, optional
    """

    db_instance_type?: str
    engine: str
    security_group_ids?: [str]
    security_ips?: [str]
    tags?: {str:str}

    check:
        isunique(security_group_ids)
        isunique(security_ips)

Package Manage Tool Updates

kpm pull supports pulling packages by package name

kpm supports pulling the corresponding package by using the kpm pull <package_name>:<package_version> command.

Taking the k8s package as an example, you can directly download the package to your local machine using the following commands:

kpm pull k8s

or

kpm pull k8s:1.27

The package downloaded with kpm pull will be saved in the directory <execution_directory>/<oci_registry>/<package_name>. For example, if you use the default kpm registry and run the kpm pull k8s command, you can find the downloaded content in the directory <execution_directory>/ghcr.io/kcl-lang/k8s.

$ tree ghcr.io/kcl-lang/k8s -L 1

ghcr.io/kcl-lang/k8s
├── api
├── apiextensions_apiserver
├── apimachinery
├── kcl.mod
├── kcl.mod.lock
├── kube_aggregator
└── vendor

6 directories, 2 files

kpm supports adding local paths as dependencies

"Different projects have different KCL packages, and there are dependencies between them. However, they are stored in different directories. I hope that these packages stored in different directories can be managed together, rather than having to put them together for them to compile." If you also have this need, you can try this feature. The kpm add command currently supports adding local paths as dependencies to a KCL package. You just need to run the command kpm add <local_package_path>, and your local package will be added as a third-party library dependency to the current package.

kpm pull k8s

After completion, you can find the downloaded k8s package in the directory "the_directory_where_you_executed_the_command/ghcr.io/kcl-lang/k8s". Create a new KCL package using the kpm init mynginx command.

kpm init mynginx

Then, navigate into this package.

cd mynginx

Inside this package, you can use the kpm add command to add the k8s package you downloaded locally as a third-party library dependency to mynginx.

kpm add ../ghcr.io/kcl-lang/k8s/

Next, add the following content to main.k.

import k8s.api.core.v1 as k8core

k8core.Pod {
    metadata.name = "web-app"
    spec.containers = [{
        name = "main-container"
        image = "nginx"
        ports = [{containerPort: 80}]
    }]
}

Normal compilation can be performed through the kpm run command.

kpm run

kpm adds checking for existing package tags

We have added a check for duplicate tags in the kpm push command. In order to avoid situations where packages with the same tag have different content, we have added restrictions on the push function in the kpm. If the version of the kcl package you push already exists, you will not be able to push the current kcl package. You will receive the following information:

kpm: package 'my_package' will be pushed.
kpm: package version '0.1.0' already exists

Modifying the content of a package that has already been pushed to the registry without changing the tag carries a high risk, as the package may already be in use by others. Therefore, if you need to push your package, we recommend:

• Change your tag and follow semantic versioning conventions.
• If you must modify the content of a package without changing the tag, you will need to delete the existing tag from the registry.

Integrations

Helmfile KCL Plugin

Helmfile is a declarative specification and tool for deploying Helm Charts. With the Helmfile KCL plugin, you can:

• Edit or verify Helm Chart through non-invasive hook methods, separating the data and logic parts of Kubernetes configuration management
  • Modify resource labels/annotations, inject sidecar container configuration
  • Use KCL schema to validate resources
  • Define your own abstract application models
• Maintain multiple environment and tenant configurations elegantly, rather than simply copying and pasting.

Here is a detailed explanation using a simple example. With the Helmfile KCL plugin, you do not need to install any components related to KCL. You only need the latest version of the Helmfile tool on your local device.

We can write a helmfile.yaml file as follows:

repositories:
- name: prometheus-community
  url: https://prometheus-community.github.io/helm-charts

releases:
- name: prom-norbac-ubuntu
  namespace: prometheus
  chart: prometheus-community/prometheus
  set:
  - name: rbac.create
    value: false
  transformers:
  # Use KCL Plugin to mutate or validate Kubernetes manifests.
  - apiVersion: krm.kcl.dev/v1alpha1
    kind: KCLRun
    metadata:
      name: "set-annotation"
      annotations:
        config.kubernetes.io/function: |
          container:
            image: docker.io/kcllang/kustomize-kcl:v0.2.0
    spec:
      source: |
        [resource | {if resource.kind == "Deployment": metadata.annotations: {"managed-by" = "helmfile-kcl"}} for resource in option("resource_list").items]

In the above file, we referenced the Prometheus Helm Chart and injected the managed-by="helmfile-kcl" label into all deployment resources of Prometheus with just one line of KCL code. The following command can be used to deploy the above configuration to the Kubernetes cluster:

helmfile apply

For more use cases, please refer to https://github.com/kcl-lang/krm-kcl

KCL Operator

KCL Operator provides cluster integration, allowing you to use Access Webhook to generate, mutate, or validate resources based on KCL configuration when apply resources to the cluster. Webhook will capture creation, application, and editing operations, and execute KCLRun on the configuration associated with each operation, and the KCL programming language can be used to

• Add labels or annotations based on a condition.
• Inject a sidecar container in all KRM resources that contain a PodTemplate.
• Validating all KRM resources using KCL Schema, such as constraints on starting containers only in a root mode.
• Generating KRM resources using an abstract model or combining and using different KRM APIs.

Here is a simple resource annotation mutation example to introduce the usage of the KCL operator.

0. Prerequisites

Prepare a Kubernetes cluster like k3d the kubectl tool.

1. Install KCL Operator

kubectl apply -f https://raw.githubusercontent.com/kcl-lang/kcl-operator/main/config/all.yaml

Use the following command to observe and wait for the pod status to be Running.

kubectl get po

2. Deploy KCL Annotation Setting Model

kubectl apply -f- << EOF
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
  name: set-annotation
spec:
  # Set dynamic parameters required for the annotation modification model, here we can add the labels we want to modify/add
  params:
    annotations:
      managed-by: kcl-operator
  # Reference the annotation modification model on OCI
  source: oci://ghcr.io/kcl-lang/set-annotation
EOF

3. Deploy a Pod to Verify the Model Result

Execute the following command to deploy a Pod resource:

kubectl apply -f- << EOF
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  annotations:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80
EOF
kubectl get po nginx -o yaml | grep kcl-operator

We can see the following output:

    managed-by: kcl-operator

We can see that the Nginx Pod automatically added the annotation managed-by=kcl-operator.

In addition, besides referencing an existing model for the source field of the KCLRun resource, we can directly set KCL code for the source field to achieve the same effect. For example:

apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
  name: set-annotation
spec:
  params:
    annotations:
      managed-by: kcl-operator
  # Resource modification can be achieved with just one line of KCL code
  source: |
    items = [item | {metadata.annotations: option("params").annotations} for item in option("items")]

We have provided more than 30 built-in models, and you can find more code examples in the following link: https://github.com/kcl-lang/krm-kcl/tree/main/examples

For example

• Use the web-service model to directly instantiate the Kubernetes resources required for a web application
• Add annotations to existing k8s resources using the set-annotation model
• Use the https-only model to verify that your Ingress configuration can only be set to https, otherwise an error will be reported.

Vault Integration

In just three steps, we can use Vault to store and manage sensitive information and use it in KCL.

Firstly, we install and use Vault to store foo and bar information.

vault kv put secret/foo foo=foo
vault kv put secret/bar bar=bar

Then write the following KCL code (main.k)

apiVersion = "apps/v1"
kind = "Deployment"
metadata = {
    name = "nginx"
    labels.app = "nginx"
    annotations: {
        "secret-store": "vault"
        # Valid format:
        #  "ref+vault://PATH/TO/KV_BACKEND#/KEY"
        "foo": "ref+vault://secret/foo#/foo"
        "bar": "ref+vault://secret/bar#/bar"
    }
}
spec = {
    replicas = 3
    selector.matchLabels = metadata.labels
    template.metadata.labels = metadata.labels
    template.spec.containers = [
        {
            name = metadata.name
            image = "${metadata.name}:1.14.2"
            ports = [{ containerPort = 80 }]
        }
    ]
}

Finally, the decrypted configuration can be obtained through the vals command-line tool

kcl main.k | vals eval -f -

For more details and use cases, please refer to here

GitLab CI Integration

Using KCL, we can not only use Github Action as CI for application publishing through GitOps, but also provide GitLab CI integration in this version. Please refer to: https://kcl-lang.io/docs/user_docs/guides/ci-integration/gitlab-ci

Other Updates and Bug Fixes

See here for more updates and bug fixes.

Documents

The versioning semantic option is added to the KCL website. Currently, v0.4.x, v0.5.x and v0.6.0 versions are supported.

Community

• Thank @jakezhu9 for his contribution to converting KCL Import tools, including Terraform Provider Schema, JsonSchema, JSON, YAML, and other configuration formats/data to KCL schema/configurations 🙌
• Thank @xxmao123 for her contribution to connecting KCL LSP language server to the Idea IDE extension 🙌
• Thank @starkers for his contribution to the KCL NeoVim extension 🙌
• Thank @starkers for adding KCL installation support to mason.nvim registry 🙌
• Thank @Ekko for his contribution to the integration of KCL cloud native tools and KCL Operator 🙌
• Thank @prahalaramji for the upgrade, update, and contribution to the KCL Homebrew installation script 🙌
• Thank @yyxhero for providing assistance and support in the Helmfile KCL plugin support 🙌
• Thank @nkabir, @mihaigalos, @prahalaramji, @yamin-oanda, @dhhopen, @magick93, @MirKml, @kolloch, @steeling, and others for their valuable feedback and discussion during the past two months of using KCL. 🙌

Additional Resources

For more information, see KCL FAQ.

Resources

Thank all KCL users for their valuable feedback and suggestions during this version release. For more resources, please refer to:

• KCL Website
• Kusion Website
• KCL Repo
• Kusion Repo

See the community for ways to join us. 👏👏👏