GPT-4 Integration in Legacy Systems: Overcoming Enterprise Technical Challenges
Secure GitHub Automation with MCP Server: Step-by-Step Implementation Guide
GitHub Actions: How to Secure Secrets and Credentials in CI/CD
Introduction
CI/CD pipelines streamline software development and accelerate releases. However, they also introduce security vulnerabilities. Secrets and credentials are essential for secure automation and must be safeguarded. GitHub Actions provides a structured approach to managing these risks. This blog explores effective strategies for securing secrets in GitHub Actions.
The Risk of Exposed Secrets
Secrets are sensitive data. Think API keys, database passwords, and SSH keys. CI/CD pipelines often need them. Storing them directly in code is a huge mistake/risk. Anyone with access to the repository can see them. This is a major security vulnerability.
The Impact of Leaked Secrets
- Leading Cause of Data Breaches: Exposed secrets are one of the leading causes of security incidents. Attackers can exploit leaked credentials to gain unauthorized access to systems, manipulate data, or deploy malicious code.
- Stolen Credentials are Common: Over 80% of security breaches involve compromised credentials. Hardcoded secrets, improperly stored access tokens, and mismanaged access control increase this risk. Cybercriminals actively scan public repositories for exposed secrets, and once compromised, these credentials grant attackers unrestricted access to systems, enabling data exfiltration, service disruption, and financial fraud. Additionally, leaked credentials facilitate lateral movement within an organization’s infrastructure, allowing attackers to escalate privileges and persist undetected. Organizations that fail to secure their credentials face not only security breaches but also financial losses, legal consequences, and reputational damage.
- Compliance Violations: Many industries are governed by strict regulations that require robust secret management practices. Failure to protect sensitive information can lead to legal penalties and reputational damage.
Why Securing Secrets in GitHub Actions Matters
- Exposed credentials lead to unauthorized access.
- Hardcoded secrets can be leaked in repositories.
- Secrets in logs or artifacts can be extracted by attackers.
- Compliance requirements mandate proper secret management.
GitHub Actions Secrets: The Solution
GitHub Actions provides a secure way to store secrets. These are environment variables. They are available only to your workflows. They are encrypted at rest. They are not stored in your repository.
How to Use GitHub Actions Secrets



Using Secrets in Workflows
Secrets are accessed in workflows using the ${{ secrets.YOUR_SECRET_NAME }} syntax. Here’s an example:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Deploy
uses: some-action@v1
with:
api_key: ${{ secrets.API_KEY }}
database_password: ${{ secrets.DATABASE_PASSWORD }}
This workflow uses two secrets: API_KEY and DATABASE_PASSWORD. These are passed to the some-action action. The action can then use them.
Best Practices for Securing Secrets



1. Use GitHub Secrets
- Store sensitive values in GitHub’s built-in secrets management.
- Access them in workflows using secrets.NAME.
- Avoid exposing secrets in logs by using environment variables securely.
2. Restrict Repository and Environment Access
- Limit who can read and modify secrets.
- Use branch protection rules to prevent unauthorized changes.
- Leverage environment-specific secrets to restrict access.
3. Use OpenID Connect (OIDC) for Federated Identity
- Avoid long-lived credentials by using short-term access tokens.
- Configure cloud providers to trust GitHub’s identity.
- Reduce the risk of secret exposure through ephemeral authentication.
4. Rotate Secrets Regularly
- Implement automated secret rotation where possible.
- Use scheduled jobs or external tools to refresh tokens.
- Revoke outdated credentials to minimize risks.
5. Scan for Hardcoded Secrets
- Use GitHub Advanced Security or tools like TruffleHog.
- Prevent accidental commits of sensitive data.
- Set up pre-commit hooks to detect secrets before pushing.
6. Mask Secrets in Logs
- GitHub automatically masks secrets, but verify log outputs.
- Use ::add-mask:: to hide additional sensitive data.
- Ensure custom scripts do not print secrets inadvertently.
7. Store Secrets in Secure Vaults
- Use external secret management tools (e.g., HashiCorp Vault, AWS Secrets Manager, Azure Key Vault).
- Fetch secrets dynamically during workflow execution.
- Minimize direct exposure of credentials within workflows.
Example: Deploying to AWS
Let’s say you want to deploy to AWS. You need AWS credentials. Store these as secrets. Use the AWS CLI action.
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Deploy to S3
run: aws s3 cp ./build s3://your-bucket/
This workflow configures AWS credentials using secrets. It then deploys a build to an S3 bucket.
Advanced Secret Management
For complex projects, consider more advanced tools. HashiCorp Vault and AWS Secrets Manager are good options. These provide centralized secret management. They offer features like secret rotation and auditing.
Implementation Steps
1. Set Up GitHub Secrets
- Navigate to repository settings → Secrets.
- Add a new secret and reference it in workflows.
2. Configure OIDC for Secure Authentication
- Enable OIDC in GitHub Actions settings.
- Configure cloud IAM policies to trust GitHub’s OIDC identity.
3. Use Secret Scanning and Detection Tools
- Enable GitHub Secret Scanning.
- Integrate scanning tools in CI/CD pipelines.
4. Regularly Rotate and Revoke Credentials
- Automate secret rotation using cloud provider tools.
- Remove unused or outdated secrets.
Key Takeaways
- Protecting secrets is vital for CI/CD security.
- GitHub Actions secrets provide a secure way to store and use sensitive data.
- Follow best practices for secret management. This minimizes security risks.
- Consider advanced tools for complex projects.
Conclusion
Secrets management in GitHub Actions is crucial for security. Use GitHub secrets, OIDC, and external vaults. Automate secret rotation and scanning to minimize exposure. Secure CI/CD automation requires strict access control and proactive monitoring.
OAuth2 & OpenID Connect Authentication for Cloud Run
Cloud Run is a serverless computing platform that allows you to deploy containerized applications with automatic scaling. However, securing these applications is crucial, especially when exposing them to the internet. OAuth2 and OpenID Connect (OIDC) provide robust authentication and authorization mechanisms to protect Cloud Run services. This blog post walks through implementing OAuth2 and OIDC authentication for a Cloud Run service.
What is OAuth2 and OpenID Connect?
- OAuth2: It is a standard authorization framework, giving third-party applications limited access to a web service without revealing the user’s credentials.
- OpenID Connect (OIDC): This is an identity layer on top of OAuth2 that offers authentication in addition to authorization.
You can use OIDC to authenticate your users to gain identity tokens proving the identity of users and is thus ideal for securing Cloud Run services.
Advantages of using OAuth2 and OpenID Connect (OIDC):
1. Enhanced Security
- Normalized Authentication: OAuth2 and OIDC standard industry mechanisms for authenticating and authorizing securely.
- Token-Based Security: They use access and ID tokens, reducing the need for storing sensitive user credentials.
- Single Sign-On (SSO): OIDC enables SSO, allowing users to log in once and access multiple services.
2. Seamless Integration with Identity Providers
- Works with Google Identity, Okta, Auth0, Azure AD, and other OAuth2/OIDC providers.
- Supports federated identity, allowing users to authenticate with social logins (Google, Facebook, GitHub, etc.).
3. Simplified Authentication for Cloud Run Services
- Identity-Aware Proxy (IAP): Google Cloud’s IAP can handle OAuth2/OIDC authentication without modifying your app.
- Automatic Token Verification: Cloud Run services can easily verify Google-issued ID tokens without additional libraries.
4. Scalability and Performance
- Stateless Authentication: No need to maintain session storage since OAuth2 tokens handle authentication.
- Efficient API Access: Using OAuth2, Cloud Run can securely call APIs (e.g., Google APIs) without requiring user credentials.
5. Role-Based Access Control (RBAC)
- OIDC allows you to extract user roles and permissions from identity providers, enabling fine-grained access control.
6. Reduced Development Overhead
- Google Cloud offers built-in support for OAuth2/OIDC, reducing the need for manual authentication handling.
- Managed services like Firebase Authentication or Cloud Identity Platform can be easily integrated.
7. Improved User Experience
- Enables secure and password-less authentication with providers like Google.
- Users can authenticate across multiple applications without re-entering credentials.
8. Compatibility with Service-to-Service Authentication
- OAuth2 client credentials flow allows Cloud Run services to securely communicate with other APIs or services.
- Google Cloud Service Accounts support workload identity federation for secure access.
Authenticating with Cloud Run
1. Select an Identity Provider (IdP)
Some well-known identity providers that support OAuth2 and OIDC are as follows:
- Google Identity Platform
- Auth0
- Okta
- Microsoft Azure AD
- Keycloak
For the purpose of this tutorial, we will use Google Identity Platform as the IdP.
2. Enable Identity-Aware Proxy (IAP)
Google’s Identity-Aware Proxy (IAP) helps you limit access to your Cloud Run service using Google authentication.
Steps to Enable IAP:
1. Enable IAP in your Google Cloud project:
gcloud services enable iap.googleapis.com
What Does It Do?
- This command activates the IAP API (iap.googleapis.com) for your Google Cloud project.
2. Deploy your Cloud Run service with authentication enabled:
gcloud run deploy my-service \
--add-cloudsql-instances=my-instance \
--service-account=my-service-account@my-project.iam.gserviceaccount.com
What Does It Do?
- gcloud run deploy my-service
- Deploys a new Cloud Run service named my-service.
- If the service already exists, this command updates it.
- –add-cloudsql-instances=my-instance
- Connects the Cloud Run service to a Cloud SQL instance named my-instance.
- This is required if your application needs to access a database hosted in Cloud SQL.
- –service-account=my-service-account@my-project.iam.gserviceaccount.com
- Assigns a specific service account to the Cloud Run service.
- This service account must have appropriate IAM roles to access Cloud SQL, such as roles/cloudsql.client.
3. Configure IAP by restricting access to authorized users:
gcloud projects add-iam-policy-binding my-project \
--member=user:example@gmail.com \
--role=roles/iap.httpsResourceAccessor
What Does It Do?
- gcloud projects add-iam-policy-binding my-project
- Adds an IAM policy binding (permission) to the Google Cloud project named my-project.
- Replace my-project with your actual Google Cloud project ID.
- –member=user:example@gmail.com
- Specifies the user who will receive the permission.
- –role=roles/iap.httpsResourceAccessor
- Grants the “IAP-secured Web App User” role (roles/iap.httpsResourceAccessor).
- This allows the specified user to access web applications protected by Identity-Aware Proxy (IAP).
- Without this role, the user will be blocked by IAP when trying to access a Cloud Run, App Engine, or Compute Engine backend.
3. Implement OAuth2 in Your Application
If you are not using IAP and want to implement OAuth2 authentication manually, follow these steps:
a. Register Your Application with Google OAuth
- Go to the Google Cloud Console.
- Navigate to APIs & Services > Credentials.
- Create a new OAuth 2.0 Client ID.
- Configure the Authorized Redirect URIs (e.g., https://your-service-url/callback).
- Note down the Client ID and Client Secret.
b. Implement OAuth2 Flow in Your Application
Your application should handle the OAuth2 authorization flow:
1. Redirect Users to the Authorization URL
from flask import Flask, redirect, request
import requests
import os
app = Flask(__name__)
CLIENT_ID = os.getenv("GOOGLE_CLIENT_ID")
REDIRECT_URI = https://your-service-url/callback
AUTH_URL = https://accounts.google.com/o/oauth2/auth
@app.route("/login")
def login():
auth_url = (f"{AUTH_URL}?client_id={CLIENT_ID}&response_type=code"
f"&redirect_uri={REDIRECT_URI}&scope=openid email profile")
return redirect(auth_url)
What Does It Do?
1. Import Required Modules
from flask import Flask, redirect, request
import requests
import os
- Import all the necessary libraries.
2. Initialize Flask App
app = Flask(__name__)
- Creates an instance of a Flask web application.
3. Define OAuth2 Configuration Variables
CLIENT_ID = os.getenv("GOOGLE_CLIENT_ID")
REDIRECT_URI = https://your-service-url/callback
AUTH_URL = "https://accounts.google.com/o/oauth2/auth" - CLIENT_ID: Retrieved from environment variables (GOOGLE_CLIENT_ID). This is the unique identifier for your app registered with Google.
- REDIRECT_URI: The callback URL where Google will redirect after authentication. This must be pre-configured in the Google OAuth settings.
- AUTH_URL: The Google OAuth2 authorization endpoint, used to initiate the authentication process.
4. Define the /login Route
@app.route("/login")
def login():
auth_url = (f"{AUTH_URL}?client_id={CLIENT_ID}&response_type=code"
f"&redirect_uri={REDIRECT_URI}&scope=openid email profile")
return redirect(auth_url) - Defines a /login route that users visit to start the authentication process.
- Builds the authorization URL dynamically:
- client_id={CLIENT_ID} → Specifies the registered application.
- response_type=code → Requests an authorization code (needed for exchanging tokens).
- redirect_uri={REDIRECT_URI} → Specifies where Google should send the user after authentication.
- scope=openid email profile → Requests access to the user’s OpenID, email, and profile information.
- Redirects the user to Google’s authentication page, where they can log in and approve access.
2. Handle the Callback and Exchange Code for Tokens
TOKEN_URL = https://oauth2.googleapis.com/token
CLIENT_SECRET = os.getenv("GOOGLE_CLIENT_SECRET")
@app.route("/callback")
def callback():
code = request.args.get("code")
data = {
"code": code,
"client_id": CLIENT_ID,
"client_secret": CLIENT_SECRET,
"redirect_uri": REDIRECT_URI,
"grant_type": "authorization_code"
}
response = requests.post(TOKEN_URL, data=data)
tokens = response.json()
return tokens
What Does It Do?
1. Define the Token Endpoint and Client Secret
TOKEN_URL = https://oauth2.googleapis.com/token
CLIENT_SECRET = os.getenv("GOOGLE_CLIENT_SECRET")
- TOKEN_URL: This is the endpoint provided by Google to exchange the authorization code for access and ID tokens.
- CLIENT_SECRET: Retrieved from environment variables (GOOGLE_CLIENT_SECRET), it is used to authenticate the application during the token request.
2. Define the /callback Route
@app.route("/callback")
def callback():
code = request.args.get("code") - callback route: This is the URL where Google redirects the user after they authenticate.
- request.args.get(“code”): Extracts the code query parameter from the URL, which was sent by Google after successful login.
3. Prepare Data for Token Exchange
data = {
"code": code,
"client_id": CLIENT_ID,
"client_secret": CLIENT_SECRET,
"redirect_uri": REDIRECT_URI,
"grant_type": "authorization_code"
} - This dictionary contains the required fields for exchanging the authorization code for tokens.
4. Send a Request to Exchange the Code for Tokens
response = requests.post(TOKEN_URL, data=data)
tokens = response.json()
- requests.post(TOKEN_URL, data=data): Sends a POST request to Google’s token endpoint with the required parameters.
- response.json(): Converts the response from Google into a JSON object containing authentication tokens.
5. Return the Tokens
return tokens
- Returns the tokens received from Google, which typically include:
- access_token → Used for API access.
- id_token → Contains user identity information (JWT format).
- expires_in → Token expiration time.
- refresh_token (if offline access is enabled) → Used to get a new access token without re-authentication.
4. Validate the ID Token
Once the access and ID tokens are obtained, you should validate the ID token to ensure its authenticity:
import jwt
from google.auth.transport import requests
from google.oauth2 import id_token
@app.route(“/profile”)
def profile():
token = request.headers.get(“Authorization”).split(“Bearer “)[1]
try:
id_info = id_token.verify_oauth2_token(token, requests.Request(), CLIENT_ID)
return id_info
except Exception as e:
return {“error”: “Invalid token”}, 401
What Does It Do?
1. Import Required Modules
import jwt
from google.auth.transport import requests
from google.oauth2 import id_token
- Import all the necessary libraries.
2. Define the /profile Route
@app.route("/profile")
def profile(): - This route handles authenticated user requests to fetch profile information.
3. Extract the Token from the Request
token = request.headers.get("Authorization").split("Bearer ")[1] - Retrieves the Authorization header from the HTTP request.
The token is expected in the format:
Authorization: Bearer <id_token>
- .split(“Bearer “)[1] extracts the actual token by removing the “Bearer ” prefix.
4. Verify the ID Token
try:
id_info = id_token.verify_oauth2_token(token, requests.Request(), CLIENT_ID)
- id_token.verify_oauth2_token(token, requests.Request(), CLIENT_ID):
- Validates the token’s signature using Google’s public keys.
- Ensures the token was issued for the correct CLIENT_ID and is not expired.
- Parses the token into a dictionary containing user identity data.
5. Return the User’s Information if Verified
return id_info
- If verification is successful, the decoded ID token (JSON format) is returned, typically containing:
{
"sub": "1234567890",
"name": "John Doe",
"email": "johndoe@example.com",
"picture": https://lh3.googleusercontent.com/a-/AOh14...
} 6. Handle Invalid or Expired Tokens
except Exception as e:
return {"error": "Invalid token"}, 401
- If verification fails due to an invalid or expired token, it returns a 401 Unauthorized error.
5. Deploy Your Application to Cloud Run
After implementing authentication, deploy the application to Cloud Run:
gcloud run deploy my-auth-service --source .
Conclusion
Implementing OAuth2 and OpenID Connect authentication in Cloud Run enhances security by ensuring only authenticated users can access your services. You can use Google Identity Platform and IAP for seamless authentication or implement OAuth2 manually using an external IdP. By following these steps, you can effectively secure your Cloud Run applications while providing a smooth authentication experience for users.
Google Lighthouse: Why Every Developer Must Use This Tool?
In the current world of development, the performance of the website is a critical aspect. A fast, user-friendly, optimized website not only invite more visitors to your site but also keep them engaged. Lighthouse is an open-source tool created by Google that has become an essential part of the tools available to web developers and digital marketers striving for higher website performance. In this blog, we will learn what Lighthouse is, how it works and how it is an essential tool for web optimization.
What Is Google Lighthouse?
Google Lighthouse is an open-source tool that automatically checks web page’s quality based on the set of parameters and offers the appropriate recommendations. The tool assesses websites based on the following five broad categories:
- Performance: Lighthouse checks how fast your webpage loads and becomes interactive. It measures when content appears, when the page is fully loaded, and when it’s ready for user input, helping developers make faster, more user-friendly websites.
- Accessibility: Lighthouse checks if your site is accessible to people with disabilities by evaluating things like colour contrast, keyboard navigation, and image alt text. It also looks at form layout and overall HTML to ensure it’s easy for everyone to use, leading to better websites for all.
- Best Practices: Lighthouse’s Best Practices ensure your website is safe, up-to-date, and secure. It checks for HTTPS, updated code, and safe libraries, as well as proper permissions and compatibility across devices and browsers, leading to a faster and safer user experience.
- SEO: SEO checks in Lighthouse help improve your site’s visibility on search engines. It ensures your page is accessible to search engines, uses proper headings, has a good title and description, and is mobile-friendly. Following these tips can boost your site’s ranking and attract more organic traffic.
- Progressive Web App (PWA): PWA tests make your website behave like an app by ensuring it’s fast, reliable, and engaging. It checks offline functionality, fast loading, HTTPS security, installability, and responsiveness across devices. Following these standards helps provide a smooth app-like experience for users.
How Does Google Lighthouse Work?
Lighthouse can perform audits by approximating a user experience on your website. Here’s how it works:
- Initiating the Audit: Users can use Lighthouse through Google Chrome DevTools, the command line or Node.js. To start the audit, just open your website and then wait when the Lighthouse will analyse the page.
Example Script: Using Node.js Script
const lighthouse = require('lighthouse');
const chromeLauncher = require('chrome-launcher');
(async () => {
const chrome = await chromeLauncher.launch({ chromeFlags: ['--headless'] });
const options = { logLevel: 'info', output: 'html', onlyCategories: ['performance'], port: chrome.port };
const runnerResult = await lighthouse('https://example.com', options);
// Use results
console.log('Report is done:', runnerResult.lhr.finalUrl);
console.log('Performance score is', runnerResult.lhr.categories.performance.score * 100);
await chrome.kill();
})();
The script runs Lighthouse on the specified URL and generates performance scores and an HTML report.
- Simulating User Conditions: Lighthouse simulates how an average user might experience your website. For example, by testing it on a mid-range Android phone with a 4G internet connection. This helps you understand how your site performs under typical conditions, especially for users with slower internet speeds or less powerful devices
Example: Using Command Line
lighthouse https://example.com --throttling.cpuSlowdownMultiplier=4 --throttling-method=simulate --output html --view
C:\Users\user>lighthouse https://www.neovasolutions.com/ --throttling.cpuSlowdownMultiplier=4 --throttling-method=simulate --output html --view We're constantly trying to improve Lighthouse and its reliability.
Learn more: https://github.com/GoogleChrome/Lighthouse/blob/main/docs/error-reporting.md
May we anonymously report runtime exceptions to improve the tool over time?
We'll remember your choice, but you can also use the flag --[no-]enable-error-reporting (y/N) · false
LH:ChromeLauncher Waiting for browser... +0ms
LH:ChromeLauncher Waiting for browser... +1ms
LH:ChromeLauncher Waiting for browser... +509ms
LH:ChromeLauncher Waiting for browser... +521ms
LH:ChromeLauncher Waiting for browser... +18ms
LH:status Connecting to browser +15
LH:status Navigating to about:blank +7ms
LH:status Benchmarking machine +78ms
LH:status Preparing target for navigation mode +1s
LH:status Cleaning origin data +133ms
LH:status Cleaning browser cache +1s
LH:status Preparing network conditions +2s
LH:status Navigating to https://www.neovasolutions.com/ +822ms
- Generating Reports on Google Lighthouse: After the audit, Lighthouse prepares few pages, and at the end of each page there will be a concise conclusion about how your site navigates and where it fails. It has advantages and disadvantages for your website and provides tips. Example Report:



Why Should You Use Lighthouse for Web Optimization?
1. Improves Core Web Vitals
Core Web Vitals focus on making websites fast, easy to use, and stable while loading. Lighthouse measures these factors and gives helpful suggestions to improve them. Fixing these issues makes your site work better for visitors and can improve your Google ranking too.
2. Boosts User Experience
Performance affects the user satisfaction on the website. Long time-to-locate, and minimal visibility makes users turn away from such sites. It is a web app that detects areas of high-performance cost and accessibility problems so you can deliver a frictionless experience.
3. Enhances SEO
It will also reveal that search engines favour well optimized web sites. Lighthouse reviews a site’s metadata, responsiveness, and other SEO factors. It also offers tips to improve its search rankings.
4. Ensures Modern Standards
The web development continues to grow fast and remain standards-conscious to make it secure and compatible. Lighthouse looks for old technologies, insecure links, and low-quality coding patterns, so you will know when it’s time to improve.
5. Supports Progressive Web App (PWA) Development
We guarantee that if you are going to create a PWA, none is as useful as Lighthouse. It helps ensure that key requirements like offline mode, fast loading, and mobile-first design are properly implemented in your app.
How to Access Google Lighthouse
1. Using Chrome DevTools:
- Open the website in Chrome, right-click, select “Inspect,” go to the Lighthouse tab, and click “Generate report.”



2. Command Line Interface:
- Install Lighthouse via npm (npm install -g lighthouse).
C:\Users\user>npm install -g lighthouse
added 164 packages in 2m
11 packages are looking for funding
run `npm fund` for details
- Run audits directly from your terminal.
C:\Users\user>lighthouse https://www.neovasolutions.com/ --throttling.cpuSlowdownMultiplier=4 --throttling-method=simulate --output html --view
We're constantly trying to improve Lighthouse and its reliability.
Learn more: https://github.com/GoogleChrome/Lighthouse/blob/main/docs/error-reporting.md
May we anonymously report runtime exceptions to improve the tool over time?
We'll remember your choice, but you can also use the flag --[no-]enable-error-reporting (y/N) · false
LH:ChromeLauncher Waiting for browser... +0ms
LH:ChromeLauncher Waiting for browser... +1ms
LH:ChromeLauncher Waiting for browser... +509ms
LH:ChromeLauncher Waiting for browser... +521ms
LH:ChromeLauncher Waiting for browser... +18ms
LH:status Connecting to browser +15
LH:status Navigating to about:blank +7ms
LH:status Benchmarking machine +78ms
LH:status Preparing target for navigation mode +1s
LH:status Cleaning origin data +133ms
LH:status Cleaning browser cache +1s
LH:status Preparing network conditions +2s
LH:status Navigating to https://www.neovasolutions.com/ +822ms
LH:status Getting artifact: DevtoolsLog +19s
LH:status Getting artifact: Trace +0ms
LH:status Getting artifact: RootCauses +1ms
LH:status Getting artifact: Accessibility +1s
LH:status Getting artifact: AnchorElements +1ms
By default, Lighthouse generates an HTML report in the current working directory.
3. Google Lighthouse CI:
- Set it up by installing the Lighthouse CI CLI and configuring it in your CI workflow to run audits and generate reports.
- Check Results Lighthouse CI gives you reports with performance scores and highlights issues to fix.



In this example, the action kicked in and failed, since I have a large image on the about page.
Conclusion
Google Lighthouse is a valuable tool that helps improve your website in many ways. It checks your site’s speed, accessibility, and best practices, providing clear feedback on areas for improvement. Regular use helps keep your site fast, user-friendly, and up to date with modern web standards, ultimately contributing to a better web experience.
Integrating WhiteSource with Maven for Vulnerability Scanning
Introduction
WhiteSource scans open-source dependencies for security vulnerabilities. WhiteSource Maven Plugin finds risks and suggests fixes. Java applications using Maven can integrate it quickly. WhiteSource with Maven let you add a dependency, configure an API key, run scans & generates vulnerability reports, suggests fixes, and verifies them by re-running scans. It enhances project security by identifying threats early in the development lifecycle.
This guide is beginner-friendly but also includes insights valuable for experts. It offers step-by-step instructions to help beginners and provides advanced tips to support seasoned professionals who need in-depth security integration.
Prerequisites For WhiteSource Maven Plugin integration
- Java project with Maven.
- WhiteSource account.
- API key.
- Internet connection.
- Basic Maven knowledge.
Example
A development team working on an e-commerce platform uses Java and Maven. It integrates WhiteSource to ensure no vulnerable open-source libraries are used in the payment module. When a vulnerability is detected, the team gets actionable steps to mitigate the risk. This integration reduces manual security checks, and it streamlines the deployment process by ensuring compliance.
Adding WhiteSource Dependency
Open pom.xml. Add this plugin inside <build>:
<plugin>
<groupId>org.whitesource</groupId>
<artifactId>whitesource-maven-plugin</artifactId>
<version>23.3.1</version>
</plugin>
Save the file. Update dependencies:
mvn clean install
Check installation:
mvn whitesource:update
Example
A Spring Boot project using spring-boot-starter-web integrates WhiteSource. Each build triggers an automatic scan of all dependencies, and it prevents known vulnerabilities from slipping into production. This approach helps maintain application stability and security. It avoids runtime issues by addressing vulnerabilities during the build phase.
Configuring WhiteSource API Key
Find API Key
- Log in to WhiteSource.
- Open Admin Panel.
- Copy the API Key.
Configure API Key in Maven
Option 1: Add in settings.xml
<settings>
<profiles>
<profile>
<id>whitesource</id>
<properties>
<whitesource.apikey>Your-API-Key-Here</whitesource.apikey>
</properties>
</profile>
</profiles>
</settings>
Option 2: Pass API Key in Command Line
mvn whitesource:update -Dwhitesource.apikey=Your-API-Key-Here
Run:
mvn whitesource:update
Example
A DevOps engineer sets up the API key in settings.xml on a Jenkins server. It ensures every build job triggers a WhiteSource scan without manual intervention. It reduces human error and keeps security checks consistent. This approach is ideal for large teams with multiple code repositories.
Running WhiteSource Scan
Run:
mvn whitesource:update
WhiteSource scans dependencies. It generates a detailed report with actionable insights.
Steps
- Open terminal.
- Run the command.
- Review scan output.
- Download the report if needed.
- Share the report with stakeholders.
- Schedule periodic scans for ongoing security.
Example
A multi-module project generates individual scan reports for each module. It lets the backend and frontend teams analyze results separately. They prioritize fixes based on vulnerability severity. It enhances communication between teams by providing clear, module-specific reports.
Interpreting the Scan Report
WhiteSource identifies issues:
- Dependency list.
- Vulnerabilities by severity.
- Suggested fixes.
- License compliance issues.
Steps
- Open WhiteSource dashboard.
- Locate the latest report.
- Review vulnerabilities.
- Apply suggested changes.
- Monitor any new vulnerabilities in future scans.
- Document fixes for compliance audits.
Example
A Log4j vulnerability is flagged. The team upgrades to a patched version and it validates the fix through regression testing. It documents the fix and updates the team about potential impacts. This method ensures long-term stability and security of the application.
Fixing Vulnerabilities and Re-Scanning
Steps
1. Upgrade dependencies in pom.xml.
2. Save and refresh.
3. Run:
mvn clean install
4. Scan again:
mvn whitesource:update
5. Confirm fixes in the report.
6. Archive reports for future audits.
7. Set up alerts for new vulnerabilities.
Example
An outdated Jackson library is updated. The new scan confirms no issues. This step is automated in CI/CD pipelines, and it provides continuous security. It ensures that if a vulnerability reappears, alerts notify the team immediately. It enhances monitoring and quick response to emerging threats.
Best Practices
- automates scans in CI/CD.
- regularly updates dependencies.
- sets up vulnerability alerts.
- assigns responsibility for vulnerability management.
- documents remediation steps for future reference.
- maintains an audit trail of all scans and fixes.
- educates developers on secure coding practices.
- sets up role-based access in WhiteSource.
- integrates WhiteSource with ticketing tools like JIRA.
- involves security teams early in the development lifecycle.
Fact#1: According to GitHub, 70% of security vulnerabilities exist in transitive dependencies. WhiteSource helps detect them early.
Fact#2: A study by Synopsys shows that 84% of commercial software codebases contain open-source components. Scanning tools like WhiteSource are critical.
Advanced Features of WhiteSource with Maven



- Policy Enforcement: It blocks builds with critical vulnerabilities.
- Automated Remediation: It automatically creates pull requests with fixes.
- Custom Rules: It sets specific security policies for your project.
- Dependency Prioritization: It focuses on the highest risk dependencies first.
- CI/CD Integration: It seamlessly integrates with Jenkins, GitHub Actions, and GitLab CI.
- Multi-Language Support: It supports not just Java but also JavaScript, Python, and more.
- Comprehensive Reporting: It provides analytics for vulnerability trends over time.
WhiteSource ensures projects remain secure. It runs periodic scans, It keeps dependencies updated, It establishes clear processes for handling vulnerabilities in the development lifecycle, It reviews vulnerability reports during sprint planning to avoid delays and helps maintain compliance with industry standards by providing thorough documentation and audit trails.
Conclusion
By automating vulnerability identification and repair in open-source dependencies, WhiteSource Maven Plugin integration improves software security. Development teams can proactively handle security issues by using an organized process that includes adding dependencies, setting up the API key, doing scans, and analyzing findings. Security workflows are streamlined by WhiteSource’s sophisticated capabilities, which include automated remediation, policy enforcement, and CI/CD integration. Long-term stability and compliance are guaranteed by best practices, frequent scanning, and dependency updates. By effectively managing vulnerabilities, keeping audit trails, and adhering to industry security standards, WhiteSource helps businesses cultivate a proactive security culture and lower risks throughout the software development lifecycle.
PostgreSQL to ClickHouse in Golang: Migration Challenges & Solutions
Introduction
With the increasing demand for real-time analytics and high-performance querying, many developers are migrating from PostgreSQL to ClickHouse. While PostgreSQL is a powerful relational database, ClickHouse offers significantly faster query execution for analytical workloads. However, the migration process comes with its own set of challenges, especially when using Golang. This blog post explores the key challenges of migrating in Golang and provides practical solutions to overcome them.
Why Migrate from PostgreSQL to ClickHouse?



Before diving into challenges and solutions, let’s understand why developers switch from PostgreSQL to ClickHouse:
- Performance Boost: ClickHouse is optimized for analytical queries and can handle large datasets more efficiently than PostgreSQL.
- Columnar Storage: Unlike PostgreSQL’s row-based storage, ClickHouse’s columnar storage enables faster aggregation and filtering operations.
- Efficient Compression: ClickHouse uses advanced compression techniques, reducing storage costs.
- Scalability: It is designed for handling petabytes of data across distributed clusters.
- Better Handling of Time-Series Data: ClickHouse excels in processing time-series data with built-in aggregation functions.
Despite these advantages, migrating from PostgreSQL to ClickHouse is not straightforward.
Steps for Migration



1st Step : Schema Conversion
- Denormalize Tables: Reduce the number of joins by pre-aggregating data where possible.
- Choose the Right Storage Engine: ClickHouse provides various table engines like MergeTree, ReplacingMergeTree and CollapsingMergeTree.
- Optimize Data Types: Use LowCardinality for repeated string values and DateTime64 for timestamp precision.
- Define Sorting Keys: Unlike PostgreSQL, ClickHouse requires a well-defined primary key for optimal performance.
2nd Step : Data Migration
- Extract Data from PostgreSQL: Use pg_dump, COPY, or pg2ch for efficient data export.
- Transform Data: Convert PostgreSQL’s JSONB and array data types into ClickHouse-compatible formats.
- Load Data into ClickHouse: Use clickhouse-client for bulk inserts or Apache Kafka for real-time streaming.
- Verify Data Integrity: Run consistency checks to ensure data accuracy.
3rd Step : Query Optimization
- Rewrite Queries: Adapt PostgreSQL queries to ClickHouse’s syntax.
- Optimize Aggregations: Use ClickHouse’s built-in functions like uniqExact, quantile, and arrayJoin.
- Partition Large Tables: Partition data based on time intervals or logical categories.
- Use Materialized Views: Precompute complex queries to speed up response times.
Key Challenges and Solutions
1. Schema Differences
Challenge:
PostgreSQL follows a strict relational model with foreign keys and constraints, whereas ClickHouse is more relaxed and does not enforce constraints.
Solution:
- Avoid foreign keys and rely on JOINs when necessary.
- Define primary keys explicitly since ClickHouse requires them for MergeTree tables.
- Choose the right table engine (e.g., MergeTree, ReplacingMergeTree) based on your use case.
Example:
PostgreSQL:
CREATE TABLE users (
id SERIAL PRIMARY KEY,
name TEXT NOT NULL,
email TEXT UNIQUE
);
ClickHouse:
CREATE TABLE users (
id UInt32,
name String,
email String
) ENGINE = MergeTree()
ORDER BY id;
2. Data Migration
Challenge:
Migrating large datasets from PostgreSQL to ClickHouse without downtime is complex.
Solution:
- Use batch processing instead of migrating all data at once.
- Export data from PostgreSQL as CSV and import into ClickHouse using clickhouse-client.
- Use Apache Kafka if real-time data migration is needed.
- Write a Golang script to fetch data from PostgreSQL and insert it into ClickHouse.
Example:
Batch migration using Golang:
package main
import (
"database/sql"
"fmt"
"log"
"github.com/ClickHouse/clickhouse-go/v2"
_ "github.com/lib/pq"
)
func main() {
pgDB, _ := sql.Open("postgres", "postgresql://user:pass@localhost/dbname")
chDB, _ := sql.Open("clickhouse", "tcp://localhost:9000?database=default")
rows, _ := pgDB.Query("SELECT id, name, email FROM users")
defer rows.Close()
for rows.Next() {
var id int
var name, email string
rows.Scan(&id, &name, &email)
_, err := chDB.Exec("INSERT INTO users (id, name, email) VALUES (?, ?, ?)", id, name, email)
if err != nil {
log.Fatal(err)
}
}
fmt.Println("Migration complete!")
}
3. Query Optimization
Challenge:
Queries optimized for PostgreSQL may not work efficiently in ClickHouse due to differences in indexing, filtering, and aggregation.
Solution:
- Replace INDEX usage with ORDER BY in ClickHouse.
- Use FINAL keyword for deduplicated results.
- Convert JOIN operations to pre-aggregated tables where possible.
Example:
PostgreSQL:
SELECT COUNT(*) FROM orders WHERE created_at > NOW() - INTERVAL '7 days';
ClickHouse:
SELECT COUNT(*) FROM orders WHERE created_at > now() - INTERVAL 7 DAY;
4. Handling Transactions
Challenge:
PostgreSQL supports transactions (BEGIN, COMMIT, ROLLBACK), while ClickHouse does not.
Solution:
- Use ClickHouse’s atomic inserts to ensure data integrity.
- Store intermediate results in temporary tables before final insertions.
5. Updating and Deleting Data
Challenge:
ClickHouse does not support standard UPDATE and DELETE operations like PostgreSQL.
Solution:
- Use ALTER TABLE DELETE WHERE for deletions.
- Use ReplacingMergeTree for handling updates.
Example:
PostgreSQL:
UPDATE users SET name = 'John Doe' WHERE id = 1;
ClickHouse:
INSERT INTO users (id, name) VALUES (1, 'John Doe')
ON DUPLICATE KEY UPDATE name = 'John Doe';
6. Concurrency and Parallelism
Challenge:
Handling multiple concurrent reads and writes differs in ClickHouse due to its design.
Solution:
- Use asynchronous inserts for high throughput.
- Utilize ClickHouse’s distributed tables to scale horizontally.
7. Indexing Limitations
Challenge:
ClickHouse does not have traditional B-tree indexing like PostgreSQL, making certain queries slower.
Solution:
- Use primary key sorting via ORDER BY.
- Use materialized views to speed up frequent queries.
8. Integrating ClickHouse with Existing Golang Applications
Challenge:
Replacing PostgreSQL with ClickHouse in a Golang project requires changes to database drivers, queries, and ORM usage.
Solution:
- Use the clickhouse-go driver to handle connections.
- Refactor query logic to accommodate ClickHouse’s syntax.
- Implement fallback strategies if ClickHouse downtime affects critical operations.
Conclusion
Migrating from PostgreSQL to ClickHouse in Golang requires careful planning and execution. Key areas to focus on include schema design, data migration strategies, query optimization, and handling updates. By leveraging batch inserts, pre-aggregated tables, and ClickHouse-specific optimizations, you can achieve a seamless transition while maintaining high performance.
If you’re planning a migration, start with small datasets, test extensively, and continuously optimize queries to get the most out of ClickHouse.























