WIP: HIVE-29579: K8s operator

[ayushtkn]

What changes were proposed in this pull request?

Add Hive Operators to deploy Hive

Why are the changes needed?

To deploy Hive on K8s

Does this PR introduce any user-facing change?

No

How was this patch tested?

Deployed on Docker Desktop -> Follow Readme -> Build Hive master (should have Tez AM PR) -> Build hive image -> Then Scenario 3 in README

After building the apache/hive-4.3.0-SNPASHOT docker image

mvn clean package -pl packaging/src/kubernetes -Pkubernetes -DskipTests
cd packaging/src/kubernetes
docker build -t apache/hive-kubernetes-operator:latest .
kubectl apply -f target/classes/META-INF/fabric8/hiveclusters.hive.apache.org-v1.yml
kubectl create namespace hive-operator

kubectl apply -f config/rbac/service-account.yaml
kubectl apply -f config/rbac/cluster-role.yaml
kubectl apply -f config/rbac/cluster-role-binding.yaml
kubectl apply -f config/operator/deployment.yaml
kubectl apply -f - <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: zookeeper
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zookeeper
  template:
    metadata:
      labels:
        app: zookeeper
    spec:
      containers:
      - name: zookeeper
        image: zookeeper:3.9
        ports:
        - containerPort: 2181
---
apiVersion: v1
kind: Service
metadata:
  name: zookeeper
spec:
  selector:
    app: zookeeper
  ports:
  - port: 2181
EOF
helm repo add ozone https://apache.github.io/ozone-helm-charts/
helm install ozone ozone/ozone --version 0.2.0 --wait
kubectl exec statefulset/ozone-om -- env OZONE_CONF_DIR=/etc/hadoop ozone sh volume create /s3v
kubectl exec statefulset/ozone-om -- env OZONE_CONF_DIR=/etc/hadoop ozone sh bucket create /s3v/hive
kubectl create secret generic hive-db-secret \
  --from-literal=password=hive123

kubectl apply -f - <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:15
        env:
        - name: POSTGRES_DB
          value: metastore
        - name: POSTGRES_USER
          value: hive
        - name: POSTGRES_PASSWORD
          value: hive123
        ports:
        - containerPort: 5432
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  selector:
    app: postgres
  ports:
  - port: 5432
EOF
kubectl apply -f config/samples/hivecluster-full-ha.yaml

Running Pods:

Beeline

kubectl exec -it deployment/my-hive-hiveserver2 -- beeline -u "jdbc:hive2://my-hive-hiveserver2:10000/"

HiveServer2 UI:

ayushsaxena@Q3NW54Y0C5 kubernetes % kubectl port-forward svc/my-hive-hiveserver2 10002:10002
Forwarding from 127.0.0.1:10002 -> 10002
Forwarding from [::1]:10002 -> 10002

[zhangbutao]

I can't wait to try out this feature!!!
BTW, besides the LLAP mode, can we also execute normal Tez tasks?

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud