Skip to main content

Prerequisites

Before setting up the development environment, ensure you have:
  • Node.js: Version 18.0 or later
  • Bun: Version 1.0 or later (recommended package manager)
  • Docker & Docker Compose: Version 24.0+ with Compose V2
  • Git: Version 2.25+
  • VS Code or another IDE with TypeScript support

System Requirements

  • RAM: 8GB minimum (16GB recommended)
  • CPU: 4+ cores
  • Storage: 20GB free space
  • OS: macOS, Linux, or Windows with WSL2

Repository Setup

Clone and Initialize

# Clone the repository
git clone https://github.com/qs-geoflow/geoflow.git
cd geoflow

# Install root dependencies
bun install

# Install all workspace dependencies
bun run clean:all

Environment Configuration

# Copy environment template
cp .env.example .env

# Edit with your local settings
nano .env
Required environment variables for development:
# Database
POSTGRES_PASSWORD=password

# Convex
INSTANCE_SECRET=your-development-secret
INSTANCE_NAME=geoflow-dev

# Development settings
NODE_ENV=development
GEOFLOW_HOST_DATA_PATH=./storage/data
GEOFLOW_HOST_TEMP_PATH=./storage/temp

Development Workflow

Starting Development Services

# Start all services with hot reload
bun run dev

# Or start specific services
bun run dev:geoflow    # Frontend only
bun run dev:backend    # Backend only
bun run dev:motia      # Workflow engine only

# Start with Docker (alternative)
bun run docker:dev

Service URLs (Development)

Project Structure

Monorepo Layout

geoflow/
├── apps/
│   ├── geoflow/          # Main React application
│   ├── backend/          # Convex backend
│   ├── motia/            # Workflow engine
│   └── docs/             # Documentation
├── packages/
│   ├── worker/           # PDAL processing service
│   ├── mcp/              # AI analysis service
│   ├── shared/           # Shared utilities
│   └── config/           # TypeScript configs
├── scripts/              # Database scripts
└── storage/              # Data directories

Key Directories

  • apps/geoflow/src/: Frontend application code
  • apps/backend/convex/: Backend queries, mutations, actions
  • packages/worker/src/: Point cloud processing logic
  • apps/motia/steps/: Workflow step definitions

Development Tasks

Frontend Development

cd apps/geoflow

# Install dependencies
bun install

# Start development server
bun run dev

# Run tests
bun run test

# Format code
bun run format

# Lint code
bun run check

Backend Development

cd apps/backend

# Start development server
bun run dev

# View dashboard
convex dashboard

Worker Development

cd packages/worker

# Install dependencies
bun install

# Start development server
bun run dev

# Test processing
curl -X POST http://localhost:3002/api/process/test

Motia Development

cd apps/motia

# Install dependencies
bun install

# Start development server
bunx motia dev --port 4010

# View dashboard (if available)
open http://localhost:4010

Code Quality

Linting and Formatting

# Lint all packages
bun run lint

# Format all code
bun run format

# Check code (lint + format)
bun run check

# Type checking
bun run type-check

Testing

# Run all tests
bun run test

# Run tests for specific package
cd apps/geoflow && bun run test

# Run tests in watch mode
bun run test:watch

Pre-commit Hooks

The project uses Biome for code quality. Pre-commit hooks are configured to:
  • Run linting on staged files
  • Format code automatically
  • Run type checking
  • Execute relevant tests

Database Development

Schema Changes

# Generate new migration
convex deploy --configure



## API Development

### Convex Queries and Mutations

```typescript
// queries.ts
import { query } from "./_generated/server";

export const getWorkflows = query({
  args: {},
  handler: async (ctx) => {
    return await ctx.db.query("workflows").collect();
  },
});

// mutations.ts
import { mutation } from "./_generated/server";

export const createWorkflow = mutation({
  args: { name: v.string(), description: v.string() },
  handler: async (ctx, args) => {
    return await ctx.db.insert("workflows", args);
  },
});

HTTP Endpoints

// http.ts
import { httpRouter } from "convex/server";

const http = httpRouter();

http.route({
  path: "/api/webhook",
  method: "POST",
  handler: async (ctx, request) => {
    const body = await request.json();
    // Process webhook
    return new Response("OK");
  },
});

export default http;

Testing Strategies

Unit Tests

// Component test
import { render, screen } from "@testing-library/react";
import { WorkflowCard } from "./WorkflowCard";

test("renders workflow name", () => {
  render(<WorkflowCard workflow={{ name: "Test Workflow" }} />);
  expect(screen.getByText("Test Workflow")).toBeInTheDocument();
});

Integration Tests

// API integration test
import { ConvexReactClient } from "convex/react";
import { api } from "../convex/_generated/api";

test("creates workflow", async () => {
  const client = new ConvexReactClient(process.env.CONVEX_URL!);
  const result = await client.mutation(api.workflows.create, {
    name: "Test",
    description: "Test workflow"
  });
  expect(result).toBeDefined();
});

E2E Tests

// E2E test with Playwright
import { test, expect } from "@playwright/test";

test("user can create workflow", async ({ page }) => {
  await page.goto("http://localhost:3000");
  await page.click("text=New Workflow");
  await page.fill("[name=name]", "My Workflow");
  await page.click("text=Create");
  await expect(page.locator("text=My Workflow")).toBeVisible();
});

Debugging

Frontend Debugging

// Add debug logging
console.log("Debug info:", data);

// Use React DevTools
// Install React DevTools browser extension

// Debug Convex queries
import { useQuery } from "convex/react";
const data = useQuery(api.workflows.get, { id });
console.log("Query result:", data);

Backend Debugging

// Add logging to Convex functions
import { log } from "convex/server";

export const myMutation = mutation({
  args: { data: v.any() },
  handler: async (ctx, args) => {
    log("Processing mutation with data:", args.data);
    // ... function logic
  },
});

Docker Debugging

# View container logs
docker compose logs -f geoflow-app

# Access container shell
docker compose exec geoflow-app sh

# Debug network issues
docker compose exec geoflow-app curl http://geoflow-convex-backend:3210

# Monitor resource usage
docker stats

Performance Monitoring

Frontend Performance

// React DevTools Profiler
// Use Chrome DevTools Performance tab

// Monitor bundle size
bun run build
ls -lh dist/static/js/

Backend Performance

// Convex dashboard metrics
// View at http://localhost:6791

// Database query performance
EXPLAIN ANALYZE SELECT * FROM workflows;

System Performance

# Monitor system resources
top
htop

# Docker resource usage
docker stats

# Network monitoring
iftop

Contributing Guidelines

Code Style

  • Use TypeScript for all new code
  • Follow existing naming conventions
  • Use descriptive variable and function names
  • Add JSDoc comments for public APIs

Git Workflow

# Create feature branch
git checkout -b feature/new-feature

# Make changes and commit
git add .
git commit -m "Add new feature"

# Push and create PR
git push origin feature/new-feature

Pull Request Process

  1. Ensure all tests pass
  2. Update documentation if needed
  3. Add migration files for schema changes
  4. Get code review approval
  5. Merge to main branch

Troubleshooting

Common Issues

Port conflicts: Change ports in docker-compose.yml or stop conflicting services Node version issues: Use nvm to manage Node.js versions Permission errors: Ensure proper file permissions for storage directories Database connection issues: Check POSTGRES_PASSWORD and container networking

Getting Help

  1. Check existing issues on GitHub
  2. Review service logs: docker compose logs
  3. Test with minimal reproduction case
  4. Ask in development discussions

Reset Development Environment

# Stop all services
docker compose down

# Clean up
bun run clean:all

# Remove volumes (WARNING: deletes data)
docker compose down -v

# Reinitialize
bun install
bun run dev