ConfNG User Guide

Complete guide to using ConfNG for configuration management in Java applications.

Introduction

ConfNG is a lightweight configuration management library designed for Java projects, particularly those using TestNG. It provides a unified way to access configuration values from multiple sources with a defined precedence order and type safety.

Key Benefits

  • Unified Configuration Access: Single API for multiple configuration sources
  • Type Safety: Enum-based configuration keys with compile-time checking
  • Flexible Precedence: Configurable source priority with sensible defaults
  • High Performance: Eager resolution minimizes runtime overhead
  • Extensible: Easy to add custom configuration sources
  • Secret Management: Built-in support for secure configuration handling

Installation

Gradle

dependencies {
    implementation 'org.confng:confng:1.0.0'
}

Maven

<dependency>
    <groupId>org.confng</groupId>
    <artifactId>confng</artifactId>
    <version>1.0.0</version>
</dependency>

Requirements

  • Java 8 or later
  • TestNG (for test integration)
  • Gson (for JSON support)
  • Reflections (for auto-discovery)

Configuration Keys

Configuration keys in ConfNG are defined as enums that implement the ConfNGKey interface. This provides type safety and compile-time checking.

Basic Configuration Keys

import org.confng.api.ConfNGKey;

public enum AppConfig implements ConfNGKey {
    // Basic keys with defaults
    BROWSER("browser", "chrome"),
    BASE_URL("base.url", "http://localhost:8080"),
    TIMEOUT("timeout", "30"),
    
    // Keys without defaults
    API_KEY("api.key"),
    DATABASE_URL("database.url");
    
    private final String key;
    private final String defaultValue;
    
    AppConfig(String key) {
        this(key, null);
    }
    
    AppConfig(String key, String defaultValue) {
        this.key = key;
        this.defaultValue = defaultValue;
    }
    
    @Override
    public String getKey() {
        return key;
    }
    
    @Override
    public String getDefaultValue() {
        return defaultValue;
    }
}

Sensitive Configuration Keys

public enum SecureConfig implements ConfNGKey {
    API_SECRET("api.secret", null, true),
    DB_PASSWORD("db.password", null, true),
    JWT_SECRET("jwt.secret", null, true);
    
    private final String key;
    private final String defaultValue;
    private final boolean sensitive;
    
    SecureConfig(String key, String defaultValue, boolean sensitive) {
        this.key = key;
        this.defaultValue = defaultValue;
        this.sensitive = sensitive;
    }
    
    @Override
    public String getKey() {
        return key;
    }
    
    @Override
    public String getDefaultValue() {
        return defaultValue;
    }
    
    @Override
    public boolean isSensitive() {
        return sensitive;
    }
}

Configuration Sources

ConfNG supports multiple configuration sources out of the box. Each source is loaded and resolved during initialization for optimal performance.

Environment Variables

Environment variables are automatically available and have the highest precedence by default.

# Set environment variables
export BROWSER=firefox
export BASE_URL=https://staging.example.com
export TIMEOUT=45

System Properties

Java system properties are automatically available with second-highest precedence.

# Set system properties
java -Dbrowser=edge -Dbase.url=https://prod.example.com MyTest

Properties Files

# test.properties
browser=safari
base.url=https://test.example.com
timeout=60
parallel.threads=4
headless=true
// Load properties file
ConfNG.loadProperties("test.properties");
ConfNG.loadProperties("config/environment.properties");

JSON Files

{
  "browser": "chrome",
  "base.url": "https://api.example.com",
  "timeout": 30,
  "features": {
    "parallel": true,
    "headless": false
  }
}
// Load JSON file
ConfNG.loadJson("config.json");
ConfNG.loadJson("environments/staging.json");

Source Precedence

ConfNG resolves configuration values based on source precedence. The first source that contains a value wins.

Default Precedence Order

  1. Environment Variables (highest precedence)
  2. System Properties
  3. Properties Files (in order of loading)
  4. JSON Files (in order of loading)
  5. Custom Sources (in order of registration)
  6. Default Values (lowest precedence)

Managing Source Precedence

// Clear all sources and start fresh
ConfNG.clearSourcesAndUseDefaults();

// Add source at specific position (0 = highest precedence)
ConfNG.registerSourceAt(0, new HighPrioritySource());

// Add source at end (lowest precedence)
ConfNG.registerSource(new LowPrioritySource());

// Load files in specific order
ConfNG.loadProperties("default.properties");    // Lower precedence
ConfNG.loadProperties("environment.properties"); // Higher precedence

Example Resolution

// Given these sources:
// Environment: BROWSER=firefox
// System Property: -Dbrowser=chrome
// Properties file: browser=safari
// JSON file: "browser": "edge"
// Default: "chrome"

String browser = ConfNG.get(AppConfig.BROWSER);
// Result: "firefox" (environment variable wins)

Type Conversion

ConfNG provides automatic type conversion for common data types with proper error handling.

String Values

String browser = ConfNG.get(AppConfig.BROWSER);
String baseUrl = ConfNG.get(AppConfig.BASE_URL);

Integer Values

Integer timeout = ConfNG.getInt(AppConfig.TIMEOUT);
Integer threads = ConfNG.getInt(AppConfig.PARALLEL_THREADS);

// Handles null values gracefully
if (timeout != null) {
    driver.manage().timeouts().implicitlyWait(timeout, TimeUnit.SECONDS);
}

Boolean Values

Boolean headless = ConfNG.getBoolean(AppConfig.HEADLESS);
Boolean parallel = ConfNG.getBoolean(AppConfig.PARALLEL_EXECUTION);

// Accepts: "true", "false" (case-insensitive)
// Throws IllegalArgumentException for invalid values

Error Handling

try {
    Integer invalidNumber = ConfNG.getInt(AppConfig.TIMEOUT);
} catch (IllegalArgumentException e) {
    // Handle invalid number format
    System.err.println("Invalid timeout value: " + e.getMessage());
}

try {
    Boolean invalidBoolean = ConfNG.getBoolean(AppConfig.HEADLESS);
} catch (IllegalArgumentException e) {
    // Handle invalid boolean format
    System.err.println("Invalid boolean value: " + e.getMessage());
}

Sensitive Data Handling

ConfNG provides built-in support for handling sensitive configuration data with automatic masking for logging and display purposes.

Marking Keys as Sensitive

public enum SecureConfig implements ConfNGKey {
    API_KEY("api.key", null, true),        // Sensitive
    DB_PASSWORD("db.password", null, true), // Sensitive
    BASE_URL("base.url", null, false);     // Not sensitive
    
    // Implementation with isSensitive() method
}

Safe Display Methods

// Get actual value (use carefully)
String apiKey = ConfNG.get(SecureConfig.API_KEY);

// Get masked value for logging
String maskedKey = ConfNG.getForDisplay(SecureConfig.API_KEY);
System.out.println("API Key: " + maskedKey); // Prints: API Key: ***MASKED***

// Display all configurations safely
String allConfigs = ConfNG.getAllForDisplay(SecureConfig.values());
System.out.println(allConfigs);
// Output:
// Configuration Values:
//   api.key = ***MASKED*** (sensitive)
//   db.password = ***MASKED*** (sensitive)
//   base.url = https://api.example.com

Logging Best Practices

@BeforeClass
public void logConfiguration() {
    // Safe logging of all configuration
    logger.info("Test Configuration:\n" + 
                ConfNG.getAllForDisplay(AppConfig.values()));
    
    // Never log sensitive values directly
    // BAD: logger.info("API Key: " + ConfNG.get(SecureConfig.API_KEY));
    // GOOD: logger.info("API Key: " + ConfNG.getForDisplay(SecureConfig.API_KEY));
}

Secret Management

ConfNG provides a framework for integrating with secret management systems like AWS Secrets Manager, HashiCorp Vault, and others.

AWS Secrets Manager Integration

import software.amazon.awssdk.services.secretsmanager.SecretsManagerClient;
import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueRequest;
import software.amazon.awssdk.services.secretsmanager.model.GetSecretValueResponse;

public class AWSSecretsSource extends SecretManagerSource {
    private final SecretsManagerClient client;
    
    public AWSSecretsSource() {
        super(300000); // 5 minute cache timeout
        this.client = SecretsManagerClient.builder()
            .region(Region.US_EAST_1)
            .build();
    }
    
    @Override
    public String getName() {
        return "AWSSecretsManager";
    }
    
    @Override
    protected String fetchSecret(String secretId) throws Exception {
        GetSecretValueRequest request = GetSecretValueRequest.builder()
            .secretId(secretId)
            .build();
        
        GetSecretValueResponse response = client.getSecretValue(request);
        return response.secretString();
    }
}

// Usage
AWSSecretsSource secretSource = new AWSSecretsSource();
secretSource.addKeyMapping("db.password", "prod/database/master-password");
secretSource.addKeyMapping("api.key", "prod/external-api/key");
ConfNG.loadSecretSource(secretSource);

HashiCorp Vault Integration

public class VaultSecretsSource extends SecretManagerSource {
    private final VaultTemplate vaultTemplate;
    
    public VaultSecretsSource(VaultTemplate vaultTemplate) {
        super(600000); // 10 minute cache timeout
        this.vaultTemplate = vaultTemplate;
    }
    
    @Override
    public String getName() {
        return "HashiCorpVault";
    }
    
    @Override
    protected String fetchSecret(String secretPath) throws Exception {
        VaultResponse response = vaultTemplate.read(secretPath);
        if (response != null && response.getData() != null) {
            return (String) response.getData().get("value");
        }
        return null;
    }
}

// Usage
VaultSecretsSource vaultSource = new VaultSecretsSource(vaultTemplate);
vaultSource.addKeyMapping("jwt.secret", "secret/app/jwt-secret");
vaultSource.addKeyMapping("encryption.key", "secret/app/encryption-key");
ConfNG.loadSecretSource(vaultSource);

Caching and Performance

// Configure caching
SecretManagerSource secretSource = new CustomSecretsSource(
    300000  // Cache timeout in milliseconds (5 minutes)
);

// Check cache status
int cacheSize = secretSource.getCacheSize();
System.out.println("Cached secrets: " + cacheSize);

// Clear cache if needed
secretSource.clearCache();

Custom Configuration Sources

You can easily extend ConfNG by implementing custom configuration sources for databases, REST APIs, or other systems.

Database Configuration Source

public class DatabaseConfigSource implements ConfigSource {
    private final Map<String, String> configCache = new HashMap<>();
    private final DataSource dataSource;
    
    public DatabaseConfigSource(DataSource dataSource) {
        this.dataSource = dataSource;
        loadConfigurations();
    }
    
    @Override
    public String getName() {
        return "DatabaseConfig";
    }
    
    @Override
    public Optional<String> get(String key) {
        return Optional.ofNullable(configCache.get(key));
    }
    
    private void loadConfigurations() {
        String sql = "SELECT config_key, config_value FROM app_config WHERE active = ?";
        
        try (Connection conn = dataSource.getConnection();
             PreparedStatement stmt = conn.prepareStatement(sql)) {
            
            stmt.setBoolean(1, true);
            ResultSet rs = stmt.executeQuery();
            
            while (rs.next()) {
                configCache.put(rs.getString("config_key"), rs.getString("config_value"));
            }
            
        } catch (SQLException e) {
            throw new RuntimeException("Failed to load database configuration", e);
        }
    }
    
    public void refresh() {
        configCache.clear();
        loadConfigurations();
    }
}

REST API Configuration Source

public class RestApiConfigSource implements ConfigSource {
    private final Map<String, String> configCache = new HashMap<>();
    private final String apiUrl;
    private final String apiKey;
    
    public RestApiConfigSource(String apiUrl, String apiKey) {
        this.apiUrl = apiUrl;
        this.apiKey = apiKey;
        loadConfigurations();
    }
    
    @Override
    public String getName() {
        return "RestApiConfig";
    }
    
    @Override
    public Optional<String> get(String key) {
        return Optional.ofNullable(configCache.get(key));
    }
    
    private void loadConfigurations() {
        try {
            HttpClient client = HttpClient.newHttpClient();
            HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create(apiUrl + "/config"))
                .header("Authorization", "Bearer " + apiKey)
                .header("Accept", "application/json")
                .build();
            
            HttpResponse<String> response = client.send(request, 
                HttpResponse.BodyHandlers.ofString());
            
            if (response.statusCode() == 200) {
                JsonObject config = JsonParser.parseString(response.body()).getAsJsonObject();
                for (Map.Entry<String, JsonElement> entry : config.entrySet()) {
                    configCache.put(entry.getKey(), entry.getValue().getAsString());
                }
            }
            
        } catch (Exception e) {
            throw new RuntimeException("Failed to load REST API configuration", e);
        }
    }
}

Registering Custom Sources

// Register at end (lowest precedence)
ConfNG.registerSource(new DatabaseConfigSource(dataSource));

// Register at specific position
ConfNG.registerSourceAt(1, new RestApiConfigSource(apiUrl, apiKey));

// Register with highest precedence
ConfNG.registerSourceAt(0, new HighPrioritySource());

Performance Optimization

ConfNG is designed for high performance with eager resolution and efficient caching strategies.

Eager Resolution

ConfNG resolves all configuration values during initialization, not at runtime:

// Values are resolved once during initialization
ConfNG.loadProperties("config.properties");
ConfNG.loadJson("config.json");

// Runtime access is just a map lookup - very fast!
String value = ConfNG.get(AppConfig.BROWSER); // O(1) lookup

Manual Resolution Control

// Force immediate resolution of all values
ConfNG.resolveAllValues();

// Resolve specific keys only
Set<String> keysToResolve = Set.of("browser", "base.url", "timeout");
ConfNG.resolveAllValues(keysToResolve);

// Check resolution status
boolean isResolved = ConfNG.isResolved();
int resolvedCount = ConfNG.getResolvedValueCount();

// Refresh all values (re-resolve from sources)
ConfNG.refresh();

Performance Monitoring

@BeforeClass
public void setupConfiguration() {
    long startTime = System.currentTimeMillis();
    
    ConfNG.loadProperties("test.properties");
    ConfNG.loadJson("config.json");
    ConfNG.resolveAllValues();
    
    long endTime = System.currentTimeMillis();
    int resolvedCount = ConfNG.getResolvedValueCount();
    
    System.out.println("Configuration loaded in " + (endTime - startTime) + "ms");
    System.out.println("Resolved " + resolvedCount + " configuration values");
}

Memory Usage

// Get all resolved keys for monitoring
Set<String> resolvedKeys = ConfNG.getResolvedKeys();
System.out.println("Memory usage: " + resolvedKeys.size() + " keys cached");

// Clear and reload if needed
ConfNG.refresh(); // Clears cache and reloads

Testing with ConfNG

ConfNG integrates seamlessly with TestNG and provides utilities for test configuration management.

Base Test Class Setup

public abstract class BaseTest {
    
    @BeforeSuite(alwaysRun = true)
    public void setupConfiguration() {
        // Load configuration in order of precedence
        ConfNG.loadProperties("test-defaults.properties");
        ConfNG.loadProperties("test.properties");
        
        // Environment-specific configuration
        String environment = System.getProperty("test.env", "local");
        ConfNG.loadProperties("environments/" + environment + ".properties");
        
        // JSON configuration
        ConfNG.loadJson("test-config.json");
        
        // Force resolution for performance
        ConfNG.resolveAllValues();
        
        // Log configuration summary
        logConfigurationSummary();
    }
    
    private void logConfigurationSummary() {
        System.out.println("=== Test Configuration ===");
        System.out.println("Resolved " + ConfNG.getResolvedValueCount() + " configuration values");
        System.out.println(ConfNG.getAllForDisplay(TestConfig.values()));
        System.out.println("========================");
    }
}

Environment-Specific Testing

// test-defaults.properties
browser=chrome
timeout=30
headless=false
parallel.threads=1

// environments/ci.properties  
headless=true
parallel.threads=4
timeout=60

// environments/local.properties
headless=false
parallel.threads=1
base.url=http://localhost:8080
# Run tests with different environments
./gradlew test -Dtest.env=local
./gradlew test -Dtest.env=ci
./gradlew test -Dtest.env=staging

Data-Driven Testing

public class WebTest extends BaseTest {
    
    @DataProvider(name = "browsers")
    public Object[][] browserProvider() {
        String browsers = ConfNG.get(TestConfig.SUPPORTED_BROWSERS);
        return Arrays.stream(browsers.split(","))
            .map(browser -> new Object[]{browser.trim()})
            .toArray(Object[][]::new);
    }
    
    @Test(dataProvider = "browsers")
    public void testOnMultipleBrowsers(String browser) {
        String baseUrl = ConfNG.get(TestConfig.BASE_URL);
        Integer timeout = ConfNG.getInt(TestConfig.TIMEOUT);
        
        WebDriver driver = WebDriverFactory.create(browser);
        driver.manage().timeouts().implicitlyWait(timeout, TimeUnit.SECONDS);
        
        // Test implementation
    }
}

Configuration Validation in Tests

@Test(priority = 1)
public void validateConfiguration() {
    // Ensure required configuration is present
    assertNotNull(ConfNG.get(TestConfig.BASE_URL), "Base URL must be configured");
    assertNotNull(ConfNG.get(TestConfig.BROWSER), "Browser must be configured");
    
    // Validate configuration values
    Integer timeout = ConfNG.getInt(TestConfig.TIMEOUT);
    assertTrue(timeout != null && timeout > 0, "Timeout must be positive");
    
    // Validate URLs
    String baseUrl = ConfNG.get(TestConfig.BASE_URL);
    assertTrue(baseUrl.startsWith("http"), "Base URL must be a valid HTTP URL");
}

Best Practices

Configuration Key Organization

// Group related configuration keys
public enum DatabaseConfig implements ConfNGKey {
    URL("database.url"),
    USERNAME("database.username"),
    PASSWORD("database.password", null, true),
    MAX_CONNECTIONS("database.max.connections", "10");
    
    // Implementation...
}

public enum WebDriverConfig implements ConfNGKey {
    BROWSER("webdriver.browser", "chrome"),
    HEADLESS("webdriver.headless", "false"),
    TIMEOUT("webdriver.timeout", "30"),
    WINDOW_SIZE("webdriver.window.size", "1920x1080");
    
    // Implementation...
}

Environment-Specific Configuration

// Use consistent naming patterns
// Environment variables: UPPER_CASE_WITH_UNDERSCORES
// Properties files: lower.case.with.dots
// JSON files: camelCase or lower.case.with.dots

public enum AppConfig implements ConfNGKey {
    // Maps to: BASE_URL env var, base.url property, baseUrl JSON
    BASE_URL("base.url"),
    
    // Maps to: API_KEY env var, api.key property, apiKey JSON  
    API_KEY("api.key", null, true);
    
    // Implementation...
}

Error Handling

@BeforeClass
public void setupConfiguration() {
    try {
        ConfNG.loadProperties("required-config.properties");
    } catch (RuntimeException e) {
        throw new RuntimeException("Failed to load required configuration", e);
    }
    
    // Validate critical configuration
    validateCriticalConfiguration();
}

private void validateCriticalConfiguration() {
    List<String> missingConfig = new ArrayList<>();
    
    if (ConfNG.get(AppConfig.BASE_URL) == null) {
        missingConfig.add("base.url");
    }
    
    if (ConfNG.get(AppConfig.API_KEY) == null) {
        missingConfig.add("api.key");
    }
    
    if (!missingConfig.isEmpty()) {
        throw new RuntimeException("Missing required configuration: " + 
                                 String.join(", ", missingConfig));
    }
}

Security Best Practices

// Always mark sensitive keys as sensitive
public enum SecureConfig implements ConfNGKey {
    API_SECRET("api.secret", null, true),
    DB_PASSWORD("db.password", null, true),
    ENCRYPTION_KEY("encryption.key", null, true);
    
    // Implementation with isSensitive() = true
}

// Use secret management for production
@BeforeClass
public void setupSecrets() {
    if (isProductionEnvironment()) {
        AWSSecretsSource secretSource = new AWSSecretsSource();
        secretSource.addKeyMapping("db.password", "prod/db/master-password");
        secretSource.addKeyMapping("api.secret", "prod/api/secret-key");
        ConfNG.loadSecretSource(secretSource);
    }
}

// Never log sensitive values
private void logConfiguration() {
    // GOOD: Uses masking for sensitive values
    logger.info("Configuration: " + ConfNG.getAllForDisplay(AppConfig.values()));
    
    // BAD: Could expose sensitive values
    // logger.info("API Key: " + ConfNG.get(SecureConfig.API_SECRET));
}

Performance Best Practices

// Load configuration once in @BeforeSuite
@BeforeSuite(alwaysRun = true)
public void setupConfiguration() {
    // Load all sources
    ConfNG.loadProperties("config.properties");
    ConfNG.loadJson("config.json");
    
    // Force resolution for better performance
    ConfNG.resolveAllValues();
}

// Cache frequently used values in test classes
public class WebTest extends BaseTest {
    private static final String BASE_URL = ConfNG.get(TestConfig.BASE_URL);
    private static final Integer TIMEOUT = ConfNG.getInt(TestConfig.TIMEOUT);
    
    @Test
    public void testSomething() {
        // Use cached values instead of repeated ConfNG.get() calls
        driver.get(BASE_URL);
        driver.manage().timeouts().implicitlyWait(TIMEOUT, TimeUnit.SECONDS);
    }
}

Troubleshooting

Common Issues

Configuration Not Found

// Debug configuration resolution
System.out.println("Resolved keys: " + ConfNG.getResolvedKeys());
System.out.println("Resolved count: " + ConfNG.getResolvedValueCount());

// Check if specific key exists
String value = ConfNG.get(AppConfig.BROWSER);
if (value == null) {
    System.out.println("Browser configuration not found, using default");
}

Type Conversion Errors

try {
    Integer timeout = ConfNG.getInt(AppConfig.TIMEOUT);
} catch (IllegalArgumentException e) {
    System.err.println("Invalid timeout format: " + e.getMessage());
    // Use default or handle error
    Integer timeout = 30; // fallback
}

File Loading Issues

// Check if files exist before loading
Path configFile = Paths.get("config.properties");
if (Files.exists(configFile)) {
    ConfNG.loadProperties("config.properties");
} else {
    System.out.println("Config file not found: " + configFile.toAbsolutePath());
}

// Use try-catch for better error handling
try {
    ConfNG.loadJson("config.json");
} catch (RuntimeException e) {
    System.err.println("Failed to load JSON config: " + e.getMessage());
    // Continue with other sources or use defaults
}

Debugging Configuration

@BeforeClass
public void debugConfiguration() {
    // Enable debug logging
    System.setProperty("confng.debug", "true");
    
    // Load configuration
    ConfNG.loadProperties("test.properties");
    ConfNG.resolveAllValues();
    
    // Print debug information
    System.out.println("=== Configuration Debug ===");
    System.out.println("Is resolved: " + ConfNG.isResolved());
    System.out.println("Resolved count: " + ConfNG.getResolvedValueCount());
    System.out.println("Resolved keys: " + ConfNG.getResolvedKeys());
    System.out.println("All values:");
    System.out.println(ConfNG.getAllForDisplay(TestConfig.values()));
    System.out.println("========================");
}

Performance Issues

// Monitor configuration loading time
long startTime = System.currentTimeMillis();

ConfNG.loadProperties("large-config.properties");
ConfNG.resolveAllValues();

long loadTime = System.currentTimeMillis() - startTime;
if (loadTime > 1000) {
    System.out.println("WARNING: Configuration loading took " + loadTime + "ms");
}

// Check memory usage
Runtime runtime = Runtime.getRuntime();
long memoryBefore = runtime.totalMemory() - runtime.freeMemory();
ConfNG.resolveAllValues();
long memoryAfter = runtime.totalMemory() - runtime.freeMemory();
System.out.println("Configuration memory usage: " + (memoryAfter - memoryBefore) + " bytes");

Getting Help