fix: allow non-existent output directories with --output flag

When using the --output flag, PerfSpect now accepts and creates
directories that don't exist yet, rather than requiring them to exist
beforehand. The directory is created once, early during initialization,
to validate write permissions before data collection begins.

Changes:
- Removed existence check in initializeApplication()
- Added createOutputDir() function in root.go (single use, kept local)
- Create output directory early (after logging, before data collection)
- Only log creation when directory is actually created
- Removed redundant directory creation calls from commands
- Detects if output path exists as a file and fails with clear error

Benefits:
- Improved usability - no manual directory creation needed
- Fail-fast validation - catches permission/disk issues immediately
- No wasted data collection - validates write access before work begins
- Single point of directory creation - simpler, more maintainable
- Cleaner logs - only logs when directory is actually created
- Creates nested directories automatically (e.g., /path/to/new/dir)

Directory creation timing:
- After logging is configured (errors can be logged)
- Before app context is set and data collection starts
- Early enough to fail fast and save time

Error handling:
- Permission denied - clear error before any work
- Disk full - detected immediately
- Read-only filesystem - fails at start
- Path exists as file - caught with clear error
- Path already exists as directory - succeeds silently

Fixes #556

Signed-off-by: Jason Harper <[email protected]>
Signed-off-by: Harper, Jason M <[email protected]>

Author: harp-intel

CLAUDE.md

@@ -0,0 +1,229 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+PerfSpect is a command-line performance analysis tool for Linux systems written in Go. It collects CPU metrics, system configuration data, telemetry, flamegraphs, and lock contention analysis. The tool targets both local and remote systems via SSH.
+
+## Build Commands
+
+**First Build:**
+```bash
+builder/build.sh  # Builds dependencies and app in Docker containers
+```
+
+**Subsequent Builds:**
+```bash
+make              # Build x86_64 binary (requires Go 1.25+)
+make perfspect-aarch64  # Build ARM64 binary
+make dist         # Build distribution packages for both architectures
+```
+
+**Testing:**
+```bash
+make test         # Run all unit tests
+```
+
+**Code Quality:**
+```bash
+make format       # Format code with gofmt (run before committing)
+make check        # Run all checks: format, vet, static analysis, license, lint, vulnerabilities, tests
+```
+
+Individual checks: `check_format`, `check_vet`, `check_static`, `check_license`, `check_lint`, `check_vuln`, `check_sec`, `check_semgrep`, `check_modernize`
+
+**All Quality Checks:**
+```bash
+make check        # Run all quality checks
+```
+
+**Cleanup:**
+```bash
+make clean        # Remove binaries and build artifacts
+make sweep        # Remove temporary output directories and logs
+```
+
+## Architecture
+
+### High-Level Structure
+
+PerfSpect follows a command-target-script architecture:
+
+1. **Commands** (`cmd/`) - User-facing commands that coordinate execution
+2. **Targets** (`internal/target/`) - Abstraction for local/remote system execution
+3. **Scripts** (`internal/script/`) - Embedded scripts that collect data on targets
+
+### Core Packages
+
+**`internal/target/`** - Target abstraction layer
+- `Target` interface defines operations on local or remote systems
+- `LocalTarget` - executes commands on the local system
+- `RemoteTarget` - executes commands via SSH on remote systems
+- Handles privilege elevation, CPU detection, file operations
+- Key methods: `RunCommand()`, `GetFile()`, `PutFile()`, `CanElevatePrivileges()`
+
+**`internal/script/`** - Script execution framework
+- `ScriptDefinition` - defines scripts with metadata (name, path, architecture, etc.)
+- `script_defs.go` - contains all script definitions used by commands
+- `RunScript()` / `RunScripts()` - execute scripts on targets, handle parallel/sequential execution
+- Scripts are embedded via `//go:embed resources` and deployed to targets at runtime
+- Supports Linux Kernel Modules (LKMs) for advanced data collection
+
+**`internal/common/`** - Shared types and utilities
+- Common data structures and constants
+- Shared functionality across commands
+
+**`internal/report/`** - Report generation
+- HTML, CSV, JSON, Excel output formatters
+- Report templates and styling
+
+**`internal/progress/`** - Progress tracking
+- Progress bars and status updates for long-running operations
+
+**`internal/util/`** - Utility functions
+- File operations, string manipulation, system utilities
+
+**`internal/cpus/`** - CPU database
+- CPU family/model/stepping identification
+- Architecture-specific information
+
+### Command Structure
+
+Each command lives in `cmd/<command>/`:
+- `metrics/` - CPU core/uncore performance metrics (TMAM, PMU events)
+- `report/` - System configuration and health reports
+- `telemetry/` - System telemetry over time
+- `flame/` - Software flamegraphs via perf
+- `lock/` - Lock contention and cache-to-cache analysis
+- `config/` - System configuration modifications
+
+Commands are registered in `cmd/root.go` using Cobra framework.
+
+### Data Flow
+
+1. User invokes command (e.g., `perfspect metrics`)
+2. Command parses flags and creates target(s) from CLI args or YAML
+3. Command selects scripts from `script_defs.go` based on what data to collect
+4. Scripts are deployed to target(s) and executed (parallel when possible)
+5. Script outputs are collected and processed
+6. Reports are generated in requested formats (HTML, CSV, JSON, etc.)
+
+### Architecture-Specific Code
+
+- Event definitions: `cmd/metrics/resources/perfmon/`, `cmd/metrics/resources/legacy/`, `cmd/metrics/resources/component/`
+- Tool binaries: `internal/script/resources/x86_64/`, `internal/script/resources/aarch64/`
+- CPU-specific logic in `internal/cpus/`
+- Metadata collection in `cmd/metrics/metadata.go`
+- Summary generation in `cmd/metrics/summary.go`
+
+When adding architecture support:
+- Follow existing patterns from x86_64 and aarch64
+- Place architecture-specific code in dedicated directories
+- Use CPU family/model/stepping detection from `internal/cpus/`
+- Add event definitions for the architecture
+- Provide validation plan for new architectures
+
+## License Requirements
+
+**Every source file must include the BSD-3-Clause license header:**
+
+Go files:
+```go
+// Copyright (C) 2021-2025 Intel Corporation
+// SPDX-License-Identifier: BSD-3-Clause
+```
+
+Shell scripts and Makefiles:
+```bash
+# Copyright (C) 2021-2025 Intel Corporation
+# SPDX-License-Identifier: BSD-3-Clause
+```
+
+Run `make check_license` to verify all files have proper headers.
+
+## Development Workflow
+
+1. Make changes
+2. Run `make check` to run all quality checks
+3. Commit with `git commit -s` (Developer Certificate of Origin required)
+
+**IMPORTANT:** Do not include attributions to Claude Code or AI assistance in commit messages or PR descriptions. Commits and PRs should appear as normal human contributions without mentioning they were generated by AI.
+
+## Key Patterns
+
+### Creating a Target
+```go
+import "perfspect/internal/target"
+
+// Local target
+t, err := target.NewLocalTarget()
+
+// Remote target with credentials
+t, err := target.NewRemoteTarget(host, user, password, keyPath)
+
+// Execute command
+output, err := t.RunCommand("ls -la")
+```
+
+### Running Scripts
+```go
+import "perfspect/internal/script"
+
+// Run single script
+output, err := script.RunScript(myTarget, scriptDef, tempDir)
+
+// Run multiple scripts (parallel when possible)
+outputs, err := script.RunScripts(myTarget, scriptDefs, ignoreErrors, tempDir)
+```
+
+### Logging
+```go
+import "log/slog"
+
+slog.Info("message", "key", value)
+slog.Debug("debug info", "detail", data)
+slog.Error("error occurred", "error", err)
+```
+
+## Testing
+
+- Unit tests use standard `testing` package and `github.com/stretchr/testify`
+- Test files: `*_test.go` alongside source files
+- Use table-driven tests for multiple scenarios
+- Run specific package tests: `go test -v ./internal/target`
+- Mock external dependencies appropriately
+
+## Profiling
+
+Set `PERFSPECT_PROFILE=1` environment variable to enable CPU and memory profiling:
+```bash
+PERFSPECT_PROFILE=1 ./perfspect metrics
+go tool pprof -http=:8080 cpu.prof
+```
+
+## Remote Targets
+
+- Target specification via `--target`, `--user`, `--key`, `--password` flags
+- Multiple targets via YAML file with `--targets` flag (see `targets.yaml` example)
+- Remote user needs password-less sudo or root for full functionality
+- SSH connection handled by `internal/target/RemoteTarget`
+
+## Output Formats
+
+Commands support multiple output formats:
+- HTML (default for reports)
+- CSV (default for metrics)
+- JSON
+- Text
+- Excel (XLSX)
+
+Use `--format` flag to specify output format. Use `--output` to specify output directory.
+
+## Contributing
+
+- Open GitHub Issues for significant changes before implementation
+- Provide validation plans for architecture-specific features
+- Named maintainer required for new features
+- Sign commits with DCO: `git commit -s`
+- See CONTRIBUTING.md for full guidelines

cmd/metrics/metrics.go

@@ -842,16 +842,8 @@ func runCmd(cmd *cobra.Command, args []string) error {
 	signalMgr := newSignalManager()
 	// short circuit when --input flag is set
 	if flagInput != "" {
-		// create output directory
-		err := common.CreateOutputDir(localOutputDir)
-		if err != nil {
-			err = fmt.Errorf("failed to create output directory: %w", err)
-			fmt.Fprintf(os.Stderr, "Error: %+v\n", err)
-			cmd.SilenceUsage = true
-			return err
-		}
 		// skip data collection and use raw data for reports
-		err = processRawData(localOutputDir)
+		err := processRawData(localOutputDir)
 		if err != nil {
 			fmt.Fprintf(os.Stderr, "Error: %v\n", err)
 			slog.Error(err.Error())
@@ -1057,16 +1049,6 @@ func runCmd(cmd *cobra.Command, args []string) error {
 			createPrometheusMetrics(targetContext.metricDefinitions)
 		}
 	}
-	// create the local output directory
-	if !flagLive && !flagPrometheusServer {
-		err = common.CreateOutputDir(localOutputDir)
-		if err != nil {
-			err = fmt.Errorf("failed to create output directory: %w", err)
-			fmt.Fprintf(os.Stderr, "Error: %+v\n", err)
-			cmd.SilenceUsage = true
-			return err
-		}
-	}
 	// start the metric production for each target
 	collectOnTargetWG := sync.WaitGroup{}
 	for i := range targetContexts {

cmd/root.go

@@ -146,6 +146,30 @@ func Execute() {
 	}
 }
 
+// createOutputDir creates the output directory if it does not exist.
+// Returns true if the directory was created, false if it already existed.
+func createOutputDir(outputDir string) (bool, error) {
+	// Check if directory already exists
+	info, err := os.Stat(outputDir)
+	if err == nil {
+		// Path exists, verify it's a directory
+		if !info.IsDir() {
+			return false, fmt.Errorf("output path exists but is not a directory: %s", outputDir)
+		}
+		return false, nil // Already exists
+	}
+	// If error is not "not exists", something else is wrong
+	if !os.IsNotExist(err) {
+		return false, fmt.Errorf("failed to check output directory: %w", err)
+	}
+	// Directory doesn't exist, create it
+	err = os.MkdirAll(outputDir, 0755) // #nosec G301
+	if err != nil {
+		return false, fmt.Errorf("failed to create output directory: %w", err)
+	}
+	return true, nil // Created successfully
+}
+
 func initializeApplication(cmd *cobra.Command, args []string) error {
 	timestamp := time.Now().Local().Format("2006-01-02_15-04-05") // app startup time
 	// set output directory path (directory will be created later when needed)
@@ -213,13 +237,15 @@ func initializeApplication(cmd *cobra.Command, args []string) error {
 	}
 	// create the output directory now to fail fast if there are permission or disk space issues
 	// this validates write access before any data collection begins
-	err = common.CreateOutputDir(outputDir)
+	created, err := createOutputDir(outputDir)
 	if err != nil {
 		slog.Error("failed to create output directory", slog.String("path", outputDir), slog.String("error", err.Error()))
 		fmt.Printf("Error: failed to create output directory: %v\n", err)
 		os.Exit(1)
 	}
-	slog.Debug("output directory created", slog.String("path", outputDir))
+	if created {
+		slog.Debug("output directory created", slog.String("path", outputDir))
+	}
 	// set app context
 	cmd.Parent().SetContext(
 		context.WithValue(

internal/common/common.go

@@ -204,17 +204,9 @@ func (rc *ReportingCommand) Run() error {
 			return err
 		}
 	}
-	// we have output data so create the output directory
-	err := CreateOutputDir(outputDir)
-	if err != nil {
-		fmt.Fprintf(os.Stderr, "Error: %v\n", err)
-		slog.Error(err.Error())
-		rc.Cmd.SilenceUsage = true
-		return err
-	}
 	// create the raw report before processing the data, so that we can save the raw data even if there is an error while processing
 	var rawReports []string
-	rawReports, err = rc.createRawReports(appContext, orderedTargetScriptOutputs)
+	rawReports, err := rc.createRawReports(appContext, orderedTargetScriptOutputs)
 	if err != nil {
 		fmt.Fprintf(os.Stderr, "Error: %v\n", err)
 		slog.Error(err.Error())
@@ -289,15 +281,6 @@ func (rc *ReportingCommand) Run() error {
 	return nil
 }
 
-// CreateOutputDir creates the output directory if it does not exist
-func CreateOutputDir(outputDir string) error {
-	err := os.MkdirAll(outputDir, 0755) // #nosec G301
-	if err != nil {
-		return fmt.Errorf("failed to create output directory: %w", err)
-	}
-	return nil
-}
-
 // DefaultInsightsFunc returns the insights table values from the table values
 func DefaultInsightsFunc(allTableValues []report.TableValues, scriptOutputs map[string]script.ScriptOutput) report.TableValues {
 	insightsTableValues := report.TableValues{