Skip to main content

7 posts tagged with "Biweekly-Newsletter"

View All Tags

· 10 min read

kcl-joining-cncf-sandbox

On September 20, 2023, the KCL project was approved by the TOC of the Cloud Native Computing Foundation (CNCF) and officially became a CNCF Sandbox project.

This means that KCL has gained recognition from the cloud native open source community, ensuring the neutrality of the project and facilitating the participation of developers and partners in project construction. It is a significant step towards dynamic configuration management and automation capabilities for cloud native application delivery.

By joining CNCF as a sandbox project, the KCL community aims to attract more developers and users to contribute and further promote the mature application of the project in cloud native scenarios. In addition, joining CNCF provides KCL with an enhanced platform for collaboration and innovation. It offers an opportunity to engage with a diverse community of developers, organizations, and industry experts at the forefront of cloud native technology. We look forward to collaborating with other CNCF projects, contributing our technical expertise, and exploring the possibilities of integrating with more CNCF projects.

What is CNCF?​

CNCF, short for Cloud Native Computing Foundation, is a sub-foundation under the Linux Foundation. CNCF is dedicated to building a sustainable ecosystem for cloud native software, covering areas such as storage, computing, orchestration, scheduling, CI/CD, DevOps, service governance, and service gateways.

Kubernetes is one of the most representative projects of CNCF.

What is CNCF Sandbox Project?​

cncf-sandbox-logo

The CNCF community classifies projects into Sandbox, Incubating, and Graduated. Well-known graduated projects include Kubernetes, Prometheus, Istio, ETCD, Containerd, ArgoCD, and Helm. For a complete list of graduated and incubating projects, you can visit https://www.cncf.io/projects/.

Sandbox is a category created by CNCF to provide a beneficial and neutral home for open source projects to promote collaboration and development. Projects selected for the Sandbox are recognized by the CNCF Technical Oversight Committee (TOC) and have the potential for experimentation and development. You can find the list of Sandbox projects at https://www.cncf.io/sandbox-projects/. To enter the Sandbox, at least 66% of the TOC members (all 11 people listed at https://github.com/cncf/toc#members) need to vote in favor, which means at least 8 people.

What is KCL?​

KCL is an open-source, constraint-based record and function language. It aims to improve the writing of complex configurations, such as cloud-native Kubernetes configurations, using mature programming language techniques and practices. KCL focuses on modularity, extensibility, and stability around configuration, aiming to create a simpler logic writing experience and build a simpler path for automation and ecosystem integration.

Key milestones of the project include:

  • May 2022: KCL was initiated and officially open-sourced by Ant Group.
  • June 2023: KCL became an official CNCF Landscape project.
  • September 2023: KCL was reviewed by the CNCF Application Delivery TAG and approved through TOC voting, officially becoming a CNCF Sandbox project (https://github.com/cncf/sandbox/issues/48).

Why do we need KCL?​

Just like recording music with a staff and storing time series data in a sequential database, we use specialized configuration and policy languages to write and manage large-scale complex configurations and policies within specific problem domains of cloud native configuration and automation. Unlike high-level general-purpose languages with hybrid writing paradigms and hybrid engineering capabilities, the core logic of specialized languages is to solve domain problems with almost infinite changes and complexity through a converged finite set of syntax and semantics, and to deposit complex configuration and strategy writing ideas and methods into language characteristics.

In addition, KCL hopes to fill the gap in configuration languages and tools in the field of lightweight client cloud native dynamic configuration through more modern declarative configuration languages and tools, and address the following issues:

  • Configuration Bloat: Most static configurations such as Kubernetes YAML in the cloud native domain need to be configured separately for each environment; In the worst-case scenario, it may introduce difficult to debug errors involving environmental cross linking, with poor stability and scalability.
  • Configuration Drift: There is often no standard way to manage the static configuration of applications and infrastructure in different environments. Using non-standard methods such as combining scripts and glue code can lead to exponential complexity growth and configuration drift.
  • Cognitive loading: Kubernetes and others, as platform technology tools for building platforms, excel in the details of the underlying unified infrastructure, but lack higher-level abstraction for application software delivery, which has a high cognitive loading on ordinary developers and affects the software delivery experience of higher-level application developers.

In response to the above issues, KCL expects to provide the following capabilities:

  • Shielding the details and complexity of infrastructure and platforms through methods such as code abstraction, reducing the cognitive loading on developers.
  • Mutate and Validate existing inventory configurations or templates to directly solve cloud generated small configuration scenarios such as Helm Chart configuration hard coding issues, but it goes far beyond that.
  • Improve team collaboration efficiency by managing large-scale configuration data across teams without side effects through language configuration.

Specifically, KCL can

  • Improve the ability of configuration semantic verification at the code level, such as schema definition, field optional/mandatory, type, range, and other configuration check and verification capabilities.
  • Provide the ability to write, combine, and abstract **configuration blocks, such as structural definition, structural inheritance, constraint definition, and configuration policy merging.
  • Improve configuration flexibility through modern programming languages and writing code, such as conditional statements, loops, functions, package management, and other features to enhance configuration reuse capabilities.
  • Provide comprehensive tool chain support, rich IDE plugins, languages, and ecological tool chain support to reduce the threshold of getting started and improve the user experience.
  • By using package management tools and OCI registry, configurations can be shared, propagated, and delivered in a simpler way among different teams/roles.
  • Provide high-performance compilers to meet the requirements of large-scale configuration scenarios, such as meeting the rendering performance requirements of generating configurations of different environments and topologies based on deployment context from a baseline configuration, as well as the performance requirements of automatic configuration modification.
  • By enhancing its automation and integration capabilities through means such as multilingual SDK and KCL language plugin, it can significantly reduce the learning cost of KCL while leveraging the value of configuration and policy writing.

In addition to the language itself, KCL also provides many additional tools such as formatting, testing, documentation, etc. to help you use, understand, and check the written configuration or strategy; Reduce the cost of configuration writing and sharing through IDE plugins such as VS Code, package management tools, and Playground; Automatically manage and execute configurations through Rust, Go, and Python multilingual SDKs.

What can KCL do?​

Dynamic Configuration Management​

standalone-kcl-form

As a configuration language, the most important feature that KCL provides to application and platform developers/SREs is dynamic configuration management. Through code abstraction, we can build an application-centric model that shields complex infrastructure and platform concepts, providing developers with a centralized and easy-to-understand interface. Additionally, KCL allows platform personnel to quickly extend and define their own models, which can be shared and reused through the OCI registry.

krm-kcl-form

Furthermore, KCL also supports direct integration with the Kubernetes Resource Model (KRM) specification. KRM KCL is a generic configuration model specification used to describe and manage various cloud native resources, such as container, pod, and service configurations and abstractions. The KRM KCL specification provides a unified way to define and manage these resources, enabling them to be portable and reusable across different environments. It operates in a fully open Kubernetes world, with minimal binding to any orchestration/engine tools or Kubernetes controllers. It allows platform personnel to extend their abstractions, configuration editing and validation logic, while providing a developer-friendly configuration management interface based on the separation of concerns.

GitOps​

gitops

Whether using standalone KCL or KRM KCL configuration forms, we support integration between KCL and various CI/CD and GitOps tools. KCL allows developers to define the resources required by applications in a declarative manner. By combining KCL with GitOps tools, we can better achieve Infrastructure as Code (IaC), improve deployment efficiency, and simplify application configuration management.

With GitOps, developers and operations teams can manage application deployments by separately modifying application and configuration code. The GitOps toolchain can automatically make changes to the configuration based on the automation capabilities of KCL, enabling continuous deployment and ensuring consistency. If any issues arise, the GitOps toolchain can quickly roll back the changes.

Integrations​

integration

In addition to integrating with GitOps automation tools like ArgoCD, as a CNCF project, KCL has also integrated with many other CNCF ecosystem projects. For example, it provides KCL plugins for existing CNCF ecosystem configuration management tools such as Helm, Kustomize, and kpt. It also provides a KCL Kubernetes Operator at runtime to meet different configuration management needs. Furthermore, we offer the following integration support:

  • Multi-language support: We provide multi-language SDKs to help users operate KCL in different languages and integrate it into their own applications.
  • Package management support: We provide the KPM package management tool to distribute and reuse KCL configurations through Docker Hub, GitHub container registry, etc.
  • Schema migration support: We support one-click migration of schemas from other ecosystems to KCL schemas, such as Go/Rust struct definitions, JsonSchema, Protobuf, OpenAPI, Terraform Provider Schema, etc.

Practices​

practice-krm-kcl

Firstly, as a small language in the cloud-native field, KCL can be directly used to solve simple problems in scenarios. For example, using the KCL model to directly inject environment variables for Kubernetes resources, and using the KCL model and Helm KCL plugins to non-invasively handle the hard-coded configuration of Helm Charts instead of directly modifying the Helm Chart by forking it.

practice-konfig-gitops

Secondly, KCL can also be used in conjunction with various CI/CD and application configuration delivery engines such as KusionStack within enterprises to achieve separation of concerns, application-centric programmable model interfaces, and GitOps processes. This simplifies the deployment and operation of scaled applications in today's hybrid multi-cloud environments, improving release and operation efficiency and developer experience.

Of course, KCL can solve more problems and scenarios than mentioned above. We will continue to share best practices from adopters in the community, and we welcome everyone to join our community for further discussions and exchanges ❤️. https://github.com/kcl-lang/community

Community​

community

In just one year of KCL being open source, we have released many versions and built the KCL community in collaboration with contributors and maintainers from all over the world. We have also gained recognition from some adopters, such as Youzan and Huawei. By joining CNCF, our goal is to increase project visibility and drive community adoption and participation, as strong and well-known foundation organizations are crucial for advancing the development of language ecosystems.

Additionally, we have received recognition from companions in the open-source community from all over the world, including China, North America, Europe, and Australia. We thank all the users and community developers who have accompanied KCL on this journey, and we welcome more companions to join our community and build together ❤️.

Conclusion​

For KCL, joining CNCF does not mean the end, but a new beginning. We will work together with our community companions to build a better KCL language, toolchain, and IDE experience! Lastly, we welcome everyone to join our community for discussions and contributions 👏👏👏

Resources​

· 5 min read

KCL is an open-source, constraint-based record and functional language that enhances the writing of complex configurations, including those for cloud-native scenarios. With its advanced programming language technology and practices, KCL is dedicated to promoting better modularity, scalability, and stability for configurations. It enables simpler logic writing and offers ease of automation APIs and integration with homegrown systems.

This section will update the KCL language community's latest developments every two weeks, including features, website updates, and the latest community news, helping everyone better understand the KCL community!

KCL Website: https://kcl-lang.io

Overview​

Thank you to all contributors for their outstanding work over the past two weeks (08.10-08.23 2023). Here is an overview of the key content:

🔧 Language and Toolchain Updates

  • KCL Import Tool Updates - Supports exporting JSON/YAML data to KCL configuration.
  • KCL IDE Updates - Supports right-click formatting ability, formatting individual files or parts of KCL code.
  • KCL Documentation Tool Updates - Exported documents support HTML escape.
  • KCL Package Management Tool KPM Updates - kpm run command execution and error message optimization, supports running KCL packages located in local paths.
  • KCL Language Updates - Optimized system package type check error messages and unified error message codes.

📰 Official Website and Use Case Updates

Special Thanks​

The following are listed in no particular order:

  • Thanks to @jakezhu9 for the contribution of converting JSON and YAML configuration data to KCL configuration in the KCL Import Tool 🙌 https://github.com/kcl-lang/kcl-go/pull/141
  • Thanks to @xxmao123 and @starkers for their contributions to the KCL NeoVim and Idea IDE extensions 🙌 https://github.com/kcl-lang/intellij-kcl/pull/12
  • Thanks to @kolloch, @prahaladramji, and others for their valuable feedback and discussions during the use of KCL in the past two weeks 🙌

Congratulations @jakezhu9 for becoming a KCL community Maintainer 🎉

KCL Operator​

KCL Operator provides cluster integration, allowing you to use Access Webhook to generate, mutate, or validate resources based on KCL configuration when apply resources to the cluster. Webhook will capture creation, application, and editing operations, and execute KCLRun on the configuration associated with each operation, and the KCL programming language can be used to

  • Add labels or annotations based on a condition.
  • Inject a sidecar container in all KRM resources that contain a PodTemplate.
  • Validating all KRM resources using KCL Schema, such as constraints on starting containers only in a root mode.
  • Generating KRM resources using an abstract model or combining and using different KRM APIs.

With KCL Operator, you can automate resource configuration management and security validation in a Kubernetes cluster using lightweight KCL code, without the need to develop a webhook server to dynamically mutate and validate configurations at runtime.

Furthermore, leveraging KCL's modeling and abstraction capabilities, we can define functionality abstractions/compositions for different resource APIs and expose them in the form of KCL Schema. We can further generate OpenAPI Schema definitions from KCL Schema for other clients in the cluster to use, without manually maintaining complex OpenAPI Schema definitions for API abstractions/compositions. Here is an example of using KCL Operator to modify resource annotations:

0. Prerequisites​

Prepare a Kubernetes cluster like k3d the kubectl tool.

1. Install KCL Operator​

kubectl apply -f https://raw.githubusercontent.com/kcl-lang/kcl-operator/main/config/all.yaml

Use the following command to observe and wait for the pod status to be Running.

kubectl get po

2. Deploy KCL Annotation Setting Model​

kubectl apply -f- << EOF
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
  name: set-annotation
spec:
  # Set dynamic parameters required for the annotation modification model, here we can add the labels we want to modify/add
  params:
    annotations:
      managed-by: kcl-operator
  # Reference the annotation modification model on OCI
  source: oci://ghcr.io/kcl-lang/set-annotation
EOF

3. Deploy a Pod to Verify the Model Result​

Execute the following command to deploy a Pod resource:

kubectl apply -f- << EOF
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  annotations:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80
EOF
kubectl get po nginx -o yaml | grep kcl-operator

We can see the following output:

    managed-by: kcl-operator

We can see that the Nginx Pod automatically added the annotation managed-by=kcl-operator.

In addition, besides referencing an existing model for the source field of the KCLRun resource, we can directly set KCL code for the source field to achieve the same effect. For example:

apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
metadata:
  name: set-annotation
spec:
  params:
    annotations:
      managed-by: kcl-operator
  # Resource modification can be achieved with just one line of KCL code
  source: |
    items = [item | {metadata.annotations: option("params").annotations} for item in option("items")]

We have provided more than 30 built-in models, and you can find more code examples in the following link: https://github.com/kcl-lang/krm-kcl/tree/main/examples

IDE Extension Updates​

In the past two weeks, we have integrated the KCL language server LSP into NeoVim and Idea, enabling the completion, navigation, and hover features supported by VS Code IDE in NeoVim and IntelliJ IDEA.

  • NeoVim KCL Extension

kcl.nvim

  • IntelliJ Extension

intellij

For more information on downloading, installation, and features of the IDE plugins, please refer to:

Resources​

❤️ Thanks to all KCL users and community members for their valuable feedback and suggestions in the community.

For more resources, please refer to