DeepFactor provides several ways of deploying the DeepFactor Portal on premises.
The following article describes how to deploy DeepFactor portal in your Kubernetes
cluster including Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), Google Kubernetes Engine (GKE), Rancher, etc.
Prerequisites
- A Kubernetes cluster (recommended version 1.18 or higher) with the following capacity:
- Three node cluster with 2X m5.large (8GB) and 1x m5.xlarge (16GB) for running DAST scans.
If you do not have a K8s cluster, you can choose to deploy DeepFactor using the OVA or AMI.
Some relevant links are placed below for quick reference.
AWS - https://docs.aws.amazon.com/eks/latest/userguide/create-cluster.html
Azure - https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal
GCP - https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-zonal-cluster#console
- Helm3 and kubectl installed on your local machine from which you intend to install DeepFactor Portal
https://helm.sh/docs/intro/install/
https://docs.docker.com/engine/install/ - kube-config for your K8s cluster on your local machine.
- An SSL certificate for the domain you want to assign to the DeepFactor portal. For example, deepfactor.mycompany.com. If you want to generate and use a self-signed certificate, you can choose to do so, during the course of installation.
Installation Steps
Download and run DeepFactor portal installation interactive script
bash -c "$(curl -L https://static.deepfactor.io/scripts/public/df-portal/installer/df-portal-installer.sh)"
You will be asked the following parameters during the course of the installation
1. Hostname: DeepFactor Portal will use this hostname. If you choose to provide your own certificate, it should be valid for this hostname. If you choose to generate a self-signed certificate, this value will be set in 'Common Name'.
Provide hostname for the portal : staging.deepfactor.io
2. Certificate: You can choose to generate a self-signed certificate or provide your own.
Do you want to continue with a Self Signed Certificate? [Y/N]?
If you choose 'N', then you will be prompted to provide the following
Provide portal key file path : ./my_private_key.pem
Provide portal crt file path : ./my_crt.pem
Provide portal ca crt file path : ./my_ca_crt.pem
If you choose 'Y', the script will download the required files and generate a self-signed certificate for your portal.
3. Image Registry: By default, DeepFactor portal images are pulled from public.ecr.aws/deepfactor/. If you intend to use your own registry, then you can provide the registry path. If you are using a registry which requires authentication, then please provide the path of the docker config file.
Provide image registry name if you want to update it, default (public.ecr.aws/deepfactor/): index.docker.io/ Provide docker config file path if you want to create image pull secret :
4. Portal admin user details
First name: Please enter the first name of the portal admin user
Last name: Please enter the last name of the portal admin user
Email address: Please enter the email address of the portal admin. You will need this to login to the portal once it is setup.
Password: Please enter a password for the admin user. This value will not be displayed on the console. Please remember this as you will need this to login to the portal once it is setup.
Provide first name of admin :
Provide last name of admin :
Provide email for admin :
Provide password for admin : \n
Re-enter the admin password : \n
5. TTL in days: This is the number of days for which the telemetry will be retained in the portal DB. Please note the alerts raised will be preserved forever, only the telemetry events received from your running applications will be deleted at the end of the TTL period.
Provide TTL days : 180
6. Portal Token: You will need to signup on my.deepfactor.io to get this token
Provide portal token (get it from https://my.deepfactor.io):
7. Memory request and limit for a DAST scan pod: When you launch a DAST scan from the portal or DeepFactor api, a job will launch a pod for the scan. The pod will terminate when the scan ends. You can set the memory request and memory limit for this pod. We recommend 8Gi memory request and 16Gi memory limit. Please hit enter to use default values.
Default and Recommended memory request of zap scan is 8Gi, provide a value to update it :
Default and Recommended memory limit of zap scan is 16Gi, provide a value to update it :
Creating override.yaml for installation.
8. Vault configuration: If you use vault for storing secrets in your Kubernetes cluster, DeepFactor can pull the JWT secret from vault. Please read Install and use Vault with DeepFactor Portal to learn more.
Please enter 'N' or hit enter to continue without vault.
Is vault configured to store secret? [y/N]? : y
Provide vault secret path : deepfactor
Provide vault role name : deepfactor
The installation script will install DeepFactor portal into your K8s cluster now using helm charts. DeepFactor pods and other resources will be created in the deepfactor namespace and the name of the helm release will be df-stable.
Portal Setup using Helm
Follow the below steps if you want to customize your portal setup. The various options available for customization can be found in the following article
Customizing your DeepFactor Portal Deployment in K8s
1. Set environment variables
Replace <deepfactor_portal_hostname> with the hostname you would like to use for the DeepFactor Portal.
export URL_ROOT="https://static.deepfactor.io/scripts/public/df-portal"
export INGRESS_HOST_NAME="<deepfactor_portal_hostname>"
export IMAGE_REGISTRY="public.ecr.aws/deepfactor/"
2. Generate self-signed certificate using helper scripts
wget $URL_ROOT/cert-gen/generate-cert.sh -O generate-cert.sh
wget $URL_ROOT/cert-gen/openssl-portal.cnf -O openssl-portal.cnf
wget $URL_ROOT/cert-gen/openssl-portalca.cnf -O openssl-portalca.cnf
chmod +x generate-cert.sh
./generate-cert.sh $INGRESS_HOST_NAME
3. Create Kubernetes secret from the above generated certificate
export PORTAL_KEY_PATH=./portal.key
export PORTAL_CERT_PATH=./portal.crt
export PORTAL_CA_CERT_PATH=./portalca.crt
kubectl create ns deepfactor
kubectl create secret tls df-certs-ingress -n deepfactor \ --key $PORTAL_KEY_PATH \ --cert $PORTAL_CERT_PATH
kubectl create secret generic deepfactor-certs -n deepfactor \ --from-file=portal.crt=$PORTAL_CERT_PATH \ --from-file=portal.key=$PORTAL_KEY_PATH \ --from-file=portalca.crt=$PORTAL_CA_CERT_PATH
4. Optional: If you want to use a private docker registry which requires authentication, you can use the following commands to generate a k8s secret. You can skip this step if you want to pull the images from DeepFactor public ECR.
export IMAGE_PULL_SECRET_NAME=deepfactor-image-pull-secret
export IMAGE_REGISTRY="<my-internal-registry>"
export DOCKER_CONFIG_PATH=~/.docker/config.json
kubectl create secret generic $IMAGE_PULL_SECRET_NAME -n deepfactor \
--from-file=.dockerconfigjson=$DOCKER_CONFIG_PATH \
--type=kubernetes.io/dockerconfigjson
5. Add the DeepFactor helm charts repo by the following command.
helm repo add deepfactor https://static.deepfactor.io/helm-charts
Verify that the repo was successfully added by running the following command
$ helm repo ls (You should see a ouptut like the following) NAME URL deepfactor https://static.deepfactor.io/helm-charts $ helm search repo -l deepfactor (You should see a ouptut like the following) NAME CHART VERSION APP VERSION DESCRIPTION deepfactor/deepfactor 1.0.1 1.0.1 DeepFactor Portal Helm Chart deepfactor/ingress-nginx 3.7.1 0.40.2 Ingress controller for Kubernetes using NGINX a...
2. Run the following command to update the helm repo to the latest charts. This step is not required if you have just added the helm repo the first time.
$ helm repo update (You should see a ouptut like the following) ...Successfully got an update from the "deepfactor" chart repository Update Complete. ⎈Happy Helming!⎈
3. You can now install DeepFactor helm charts. You will need to supply an override.yaml file to helm install command. This file can be used to customize the DeepFactor portal deployment. You can read more about the different configurations options in the following article
Customizing your DeepFactor Portal Deployment in K8s
You can get the portalToken by signing up on my.deepfactor.io.
Create a file override.yaml with the parameters you want override. Please note dfstartup.config parameters are required
echo "
dfstartup:
config:
firstName: "<admin-first-name>"
lastName: "<admin-last-name>"
emailID: "<admin-email-id>"
ttlDays: "<ttl-days>"
portalToken: "<admin-portal-token>"
password: "<password>"
ingress:
hostName: $INGRESS_HOST_NAME
imagePullSecrets:
- name: $IMAGE_PULL_SECRET_NAME
appsettings:
numberOfConcurrentWebScansAllowed: <max-number-of-concurrent-web-scans-allowed>
" > override.yaml
$ helm install df-stable deepfactor/deepfactor -f override.yaml --namespace=deepfactor (You should see a ouptut like the following. I had given the `release-name`=beta) NAME: beta LAST DEPLOYED: Mon Apr 5 16:17:08 2021 NAMESPACE: deepfactor STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: The DeepFactor portal has been installed check its status by running: $ helm ls -n deepfactor
$ kubectl get pods -n deepfactor
where
df-stable is the release name. You can provide a different string if you would like.
5. Execute the following command to get the public IP of the portal.
$ kubectl get svc -n deepfactor ... beta-ingress-nginx-controller LoadBalancer 10.100.43.157 abcdefghijklmnopq-545188163.us-east-2.elb.amazonaws.com 80:32013/TCP,443:30366/TCP,8443:30520/TCP 7m30s ...
Find the df-stable-ingress-nginx-controller service and copy the long DNS string and set a DNS record to point <deepfactor_portal_hostname> to this domain (either using an ALIAS record in AWS or an A record).
You can now access the DeepFactor portal at <deepfactor_portal_hostname> and log in to the portal using the email address and password entered in the override.yaml.
Uninstalling DeepFactor Portal
Run the following command to uninstall the portal.
$ helm uninstall df-stable --namespace=deepfactor
Delete the postgres and clickhouse pvc to free up the space. Please note you will lose all the telemetry and alerts data.
$ kubectl get pvc --namespace=deepfactor $ kubectl delete pvc <postgres-pvc-name> <clickhouse-pvc-name> <archivestore-pvc-name> <symbolsvc-pvc-name> --namespace=deepfactor
$ kubectl get secrets -n deepfactor $ kubectl delete secrets regcred deepfactor-certs <ingress-secret-name> -n deepfactor
Comments
0 comments
Please sign in to leave a comment.