2025-12-21 00:34:04
Every app that goes global faces the same challenge: how do you update translations without forcing users to download a new version? Traditional ARB file-based localization in Flutter works well for small projects, but the moment you need to fix a typo in your Spanish translation or add support for a new market, you're stuck waiting for app store approval.
This implementation solves that problem by moving translations to a backend API. Your marketing team can update copy in real-time, you can A/B test different phrasings, and users always see the latest content without any app updates. The system includes offline support through intelligent caching, bandwidth optimization with HTTP 304 handling, and full RTL language support for markets like Arabic.
By the end of this guide, you'll have a production-ready localization system built with Clean Architecture principles and the BLoC pattern. The complete source code is available in this repository.
The localization system supports six languages out of the box: English, Spanish, French, German, Hindi, and Arabic. Each translation set includes over 50 keys covering common UI elements like buttons, form labels, error messages, and navigation items.
The architecture follows a specific flow: when the app launches, it checks for cached translations. If they exist, it sends a conditional request to the server with the cached version number. The server either returns new translations (HTTP 200) or confirms the cache is still valid (HTTP 304). This approach minimizes bandwidth usage while ensuring users always have current content.
For offline scenarios, the app falls back to cached translations automatically. Users can switch languages instantly because translations are pre-fetched and stored locally.
The codebase follows Clean Architecture with three distinct layers, each with a specific responsibility:
lib/
├── core/
│ ├── config/
│ │ ├── api_config.dart
│ │ └── app_strings.dart
│ ├── constants/
│ │ └── supported_locales.dart
│ ├── di/
│ │ └── injection_container.dart
│ ├── error/
│ │ ├── exceptions.dart
│ │ └── failures.dart
│ ├── network/
│ │ └── network_info.dart
│ └── theme/
│ └── app_theme.dart
│
└── features/
└── localization/
├── data/
│ ├── datasources/
│ │ ├── localization_remote_datasource.dart
│ │ └── localization_local_datasource.dart
│ ├── models/
│ │ └── translation_model.dart
│ └── repositories/
│ └── localization_repository_impl.dart
│
├── domain/
│ ├── entities/
│ │ └── translation_entity.dart
│ ├── repositories/
│ │ └── localization_repository.dart
│ └── usecases/
│ ├── get_translations_usecase.dart
│ ├── change_locale_usecase.dart
│ └── get_supported_locales_usecase.dart
│
└── presentation/
├── bloc/
│ ├── localization_bloc.dart
│ ├── localization_event.dart
│ └── localization_state.dart
├── pages/
│ ├── home_page.dart
│ └── settings_page.dart
└── widgets/
└── language_selector.dart
The domain layer contains pure business logic with no external dependencies. It defines what the app needs through abstract interfaces. The data layer implements those interfaces using Dio for API calls and Hive for local storage. The presentation layer uses BLoC to manage UI state and respond to user interactions.
Add these packages to your pubspec.yaml:
dependencies:
connectivity_plus: ^6.1.0
dartz: ^0.10.1
dio: ^5.7.0
equatable: ^2.0.5
flutter:
sdk: flutter
flutter_bloc: ^9.1.1
flutter_localizations:
sdk: flutter
get_it: ^7.7.0
hive_flutter: ^1.1.0
intl: ^0.20.2
shared_preferences: ^2.3.3
Each package serves a specific purpose: dio handles HTTP requests with built-in interceptor support, hive_flutter provides fast local storage for cached translations, flutter_bloc manages state predictably, get_it serves as a lightweight dependency injection container, and dartz enables functional error handling with the Either type.
Enable network access in android/app/src/main/AndroidManifest.xml:
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET"/>
<application
android:label="flutter_blog"
android:name="${applicationName}"
android:icon="@mipmap/ic_launcher"
android:usesCleartextTraffic="true">
<!-- Activity configuration -->
</application>
</manifest>
The usesCleartextTraffic="true" attribute allows HTTP connections during development. Remove this in production where all traffic should use HTTPS.
The API config centralizes all endpoint definitions and provides different base URLs for various development environments:
/// lib/core/config/api_config.dart
class ApiConfig {
ApiConfig._();
/// Android emulator maps 10.0.2.2 to host's localhost
static const String androidEmulatorUrl = 'http://10.0.2.2:3000/api/v1';
/// Direct localhost for iOS Simulator and desktop
static const String localhostUrl = 'http://localhost:3000/api/v1';
/// Your machine's WiFi IP for physical device testing
static const String physicalDeviceUrl = 'http://192.168.0.103:3000/api/v1';
/// Switch this based on your testing environment
static const String baseUrl = physicalDeviceUrl;
static const String translations = '/translations';
static const String translationsByLocale = '/translations/{locale}';
static const String supportedLocales = '/translations/supported-locales';
static const Duration connectTimeout = Duration(seconds: 30);
static const Duration receiveTimeout = Duration(seconds: 30);
static const Duration sendTimeout = Duration(seconds: 30);
static Map<String, String> get defaultHeaders => {
'Content-Type': 'application/json',
'Accept': 'application/json',
};
}
A common issue when testing on Android emulators: requests to localhost:3000 fail because the emulator's localhost points to itself, not your development machine. The special IP 10.0.2.2 solves this by mapping to the host's loopback interface.
The error handling system uses two levels: exceptions for immediate errors and failures for domain-level error representation.
/// lib/core/error/exceptions.dart
class ServerException implements Exception {
final String message;
final int? statusCode;
final dynamic responseBody;
const ServerException({
required this.message,
this.statusCode,
this.responseBody,
});
}
class CacheException implements Exception {
final String message;
const CacheException({this.message = 'Cache operation failed'});
}
/// Special exception for HTTP 304 Not Modified responses
class CacheValidException implements Exception {
const CacheValidException();
}
The CacheValidException deserves attention. When the server returns HTTP 304, it means our cached data is still current. This isn't an error condition - it's actually good news. We throw this exception to signal that the repository should use cached data instead of treating it as a failure.
/// lib/core/error/failures.dart
abstract class Failure extends Equatable {
final String message;
final String? code;
const Failure({required this.message, this.code});
@override
List<Object?> get props => [message, code];
}
class ServerFailure extends Failure {
final int? statusCode;
const ServerFailure({
required super.message,
super.code,
this.statusCode,
});
}
class NetworkFailure extends Failure {
const NetworkFailure({
super.message = 'No internet connection. Please check your network.',
super.code = 'NETWORK_ERROR',
});
}
class CacheFailure extends Failure {
const CacheFailure({
super.message = 'Failed to access local cache.',
super.code = 'CACHE_ERROR',
});
}
class LocalizationFailure extends Failure {
final String? locale;
const LocalizationFailure({
required super.message,
super.code = 'LOCALIZATION_ERROR',
this.locale,
});
}
The network info service wraps connectivity_plus to provide a clean interface for checking network status:
/// lib/core/network/network_info.dart
abstract class NetworkInfo {
Future<bool> get isConnected;
Stream<bool> get onConnectivityChanged;
}
class NetworkInfoImpl implements NetworkInfo {
final Connectivity _connectivity;
NetworkInfoImpl(this._connectivity);
@override
Future<bool> get isConnected async {
final result = await _connectivity.checkConnectivity();
return !result.contains(ConnectivityResult.none);
}
@override
Stream<bool> get onConnectivityChanged {
return _connectivity.onConnectivityChanged.map(
(result) => !result.contains(ConnectivityResult.none),
);
}
}
The core entity represents a complete translation set for a locale:
/// lib/features/localization/domain/entities/translation_entity.dart
class TranslationEntity extends Equatable {
final String locale;
final String version;
final DateTime updatedAt;
final Map<String, String> translations;
const TranslationEntity({
required this.locale,
required this.version,
required this.updatedAt,
required this.translations,
});
factory TranslationEntity.empty(String locale) {
return TranslationEntity(
locale: locale,
version: '0.0.0',
updatedAt: DateTime.now(),
translations: const {},
);
}
String get(String key, [Map<String, dynamic>? params]) {
String value = translations[key] ?? key;
if (params != null) {
params.forEach((paramKey, paramValue) {
value = value.replaceAll('{$paramKey}', paramValue.toString());
});
}
return value;
}
@override
List<Object?> get props => [locale, version, updatedAt, translations];
}
The version field is crucial for cache invalidation. When fetching translations, the app sends its current cached version to the server. If the server has a newer version, it returns the full translation set. If not, it returns HTTP 304.
The repository interface defines what operations the app needs without specifying how they're implemented:
/// lib/features/localization/domain/repositories/localization_repository.dart
abstract class LocalizationRepository {
Future<Either<Failure, TranslationEntity>> getTranslations(
String locale, {
bool forceRefresh = false,
});
Future<Either<Failure, SupportedLocalesEntity>> getSupportedLocales();
Future<Either<Failure, bool>> cacheTranslations(TranslationEntity translations);
Future<Either<Failure, TranslationEntity>> getCachedTranslations(String locale);
Future<String?> getCachedVersion(String locale);
Future<Either<Failure, bool>> clearCache();
Future<Either<Failure, bool>> savePreferredLocale(String locale);
Future<String?> getPreferredLocale();
}
Using Either<Failure, T> from dartz forces explicit error handling. Every caller must handle both success and failure cases - you can't accidentally ignore errors.
Each use case encapsulates a single business operation:
/// lib/features/localization/domain/usecases/get_translations_usecase.dart
class GetTranslationsUseCase {
final LocalizationRepository _repository;
const GetTranslationsUseCase(this._repository);
Future<Either<Failure, TranslationEntity>> call(
GetTranslationsParams params,
) async {
return _repository.getTranslations(
params.locale,
forceRefresh: params.forceRefresh,
);
}
}
class GetTranslationsParams extends Equatable {
final String locale;
final bool forceRefresh;
const GetTranslationsParams({
required this.locale,
this.forceRefresh = false,
});
@override
List<Object?> get props => [locale, forceRefresh];
}
/// lib/features/localization/domain/usecases/change_locale_usecase.dart
class ChangeLocaleUseCase {
final LocalizationRepository _repository;
const ChangeLocaleUseCase(this._repository);
Future<Either<Failure, TranslationEntity>> call(
ChangeLocaleParams params,
) async {
final translationsResult = await _repository.getTranslations(params.locale);
return translationsResult.fold(
(failure) => Left(failure),
(translations) async {
await _repository.savePreferredLocale(params.locale);
await _repository.cacheTranslations(translations);
return Right(translations);
},
);
}
}
class ChangeLocaleParams extends Equatable {
final String locale;
const ChangeLocaleParams({required this.locale});
@override
List<Object?> get props => [locale];
}
The remote data source handles all API communication. Pay close attention to the HTTP 304 handling:
/// lib/features/localization/data/datasources/localization_remote_datasource.dart
class LocalizationRemoteDataSourceImpl implements LocalizationRemoteDataSource {
final Dio _dio;
const LocalizationRemoteDataSourceImpl(this._dio);
@override
Future<TranslationModel> getTranslations(
String locale, {
String? currentVersion,
}) async {
try {
final queryParams = <String, dynamic>{};
if (currentVersion != null) {
queryParams['current_version'] = currentVersion;
}
final response = await _dio.get(
'/translations/$locale',
queryParameters: queryParams.isNotEmpty ? queryParams : null,
options: Options(
// This is the key: accept 304 as a valid status
validateStatus: (status) => status != null && (status < 400 || status == 304),
),
);
// Handle 304 Not Modified
if (response.statusCode == 304) {
throw const CacheValidException();
}
return TranslationModel.fromJson(response.data as Map<String, dynamic>);
} on CacheValidException {
rethrow;
} on DioException catch (e) {
if (e.response?.statusCode == 304) {
throw const CacheValidException();
}
throw ServerException(
message: e.message ?? 'Failed to fetch translations',
statusCode: e.response?.statusCode,
responseBody: e.response?.data,
);
}
}
}
By default, Dio treats any non-2xx status code as an error. The validateStatus option tells Dio to accept 304 as a valid response. Without this, Dio would throw an exception before we could check the status code.
The local data source uses Hive for caching and SharedPreferences for simple key-value storage:
/// lib/features/localization/data/datasources/localization_local_datasource.dart
class LocalizationLocalDataSourceImpl implements LocalizationLocalDataSource {
static const String _translationsBoxName = 'translations_cache';
static const String _preferredLocaleKey = 'preferred_locale';
final SharedPreferences _sharedPreferences;
Box<String>? _translationsBox;
LocalizationLocalDataSourceImpl(this._sharedPreferences);
Future<Box<String>> get _box async {
if (_translationsBox != null && _translationsBox!.isOpen) {
return _translationsBox!;
}
_translationsBox = await Hive.openBox<String>(_translationsBoxName);
return _translationsBox!;
}
@override
Future<void> cacheTranslations(TranslationModel translations) async {
try {
final box = await _box;
final jsonString = jsonEncode(translations.toJson());
await box.put(translations.locale, jsonString);
} catch (e) {
throw CacheException(message: 'Failed to cache translations: $e');
}
}
@override
Future<TranslationModel> getCachedTranslations(String locale) async {
try {
final box = await _box;
final jsonString = box.get(locale);
if (jsonString == null) {
throw CacheException(
message: 'No cached translations found for locale: $locale',
);
}
final json = jsonDecode(jsonString) as Map<String, dynamic>;
return TranslationModel.fromJson(json);
} catch (e) {
if (e is CacheException) rethrow;
throw CacheException(message: 'Failed to read cached translations: $e');
}
}
@override
Future<String?> getCachedVersion(String locale) async {
try {
final box = await _box;
final jsonString = box.get(locale);
if (jsonString == null) return null;
final json = jsonDecode(jsonString) as Map<String, dynamic>;
return json['version'] as String?;
} catch (e) {
return null;
}
}
@override
Future<void> savePreferredLocale(String locale) async {
await _sharedPreferences.setString(_preferredLocaleKey, locale);
}
@override
Future<String?> getPreferredLocale() async {
return _sharedPreferences.getString(_preferredLocaleKey);
}
}
The repository coordinates between remote and local data sources, implementing the offline-first strategy:
/// lib/features/localization/data/repositories/localization_repository_impl.dart
class LocalizationRepositoryImpl implements LocalizationRepository {
final LocalizationRemoteDataSource _remoteDataSource;
final LocalizationLocalDataSource _localDataSource;
final NetworkInfo _networkInfo;
const LocalizationRepositoryImpl({
required LocalizationRemoteDataSource remoteDataSource,
required LocalizationLocalDataSource localDataSource,
required NetworkInfo networkInfo,
}) : _remoteDataSource = remoteDataSource,
_localDataSource = localDataSource,
_networkInfo = networkInfo;
@override
Future<Either<Failure, TranslationEntity>> getTranslations(
String locale, {
bool forceRefresh = false,
}) async {
final isConnected = await _networkInfo.isConnected;
if (isConnected) {
return _getRemoteTranslations(locale, forceRefresh: forceRefresh);
} else {
return _getCachedOrFail(locale);
}
}
Future<Either<Failure, TranslationEntity>> _getRemoteTranslations(
String locale, {
bool forceRefresh = false,
}) async {
try {
String? currentVersion;
if (!forceRefresh) {
currentVersion = await _localDataSource.getCachedVersion(locale);
}
final remoteTranslations = await _remoteDataSource.getTranslations(
locale,
currentVersion: currentVersion,
);
await _localDataSource.cacheTranslations(remoteTranslations);
return Right(remoteTranslations);
} on CacheValidException {
// Server returned 304 - cached data is still valid
return _getCachedOrFail(locale);
} on DioException catch (e) {
if (e.type == DioExceptionType.connectionTimeout ||
e.type == DioExceptionType.receiveTimeout ||
e.type == DioExceptionType.connectionError) {
return _getCachedOrFail(locale);
}
return Left(
ServerFailure(
message: e.message ?? 'Failed to fetch translations',
statusCode: e.response?.statusCode,
),
);
} on ServerException catch (e) {
return Left(
ServerFailure(
message: e.message,
statusCode: e.statusCode,
),
);
} catch (e) {
return _getCachedOrFail(locale);
}
}
Future<Either<Failure, TranslationEntity>> _getCachedOrFail(
String locale,
) async {
try {
final cachedTranslations =
await _localDataSource.getCachedTranslations(locale);
return Right(cachedTranslations);
} on CacheException catch (e) {
return Left(
LocalizationFailure(
message: e.message,
locale: locale,
),
);
}
}
}
The key insight here is the CacheValidException catch block. When the server returns 304, we don't treat it as an error - we simply return the cached data. This is both efficient (no unnecessary data transfer) and correct (the user gets valid translations).
The BLoC uses sealed classes for type-safe event and state handling:
/// lib/features/localization/presentation/bloc/localization_event.dart
sealed class LocalizationEvent extends Equatable {
const LocalizationEvent();
@override
List<Object?> get props => [];
}
final class InitializeLocalizationEvent extends LocalizationEvent {
const InitializeLocalizationEvent();
}
final class LoadTranslationsEvent extends LocalizationEvent {
final String locale;
final bool forceRefresh;
const LoadTranslationsEvent({
required this.locale,
this.forceRefresh = false,
});
@override
List<Object?> get props => [locale, forceRefresh];
}
final class ChangeLocaleEvent extends LocalizationEvent {
final String locale;
const ChangeLocaleEvent({required this.locale});
@override
List<Object?> get props => [locale];
}
final class RefreshTranslationsEvent extends LocalizationEvent {
const RefreshTranslationsEvent();
}
/// lib/features/localization/presentation/bloc/localization_state.dart
sealed class LocalizationState extends Equatable {
const LocalizationState();
@override
List<Object?> get props => [];
}
final class LocalizationInitial extends LocalizationState {
const LocalizationInitial();
}
final class LocalizationLoading extends LocalizationState {
final String? message;
const LocalizationLoading({this.message});
@override
List<Object?> get props => [message];
}
final class LocalizationLoaded extends LocalizationState {
final Locale locale;
final TranslationEntity translations;
final List<String> supportedLocales;
const LocalizationLoaded({
required this.locale,
required this.translations,
this.supportedLocales = const ['en', 'es', 'fr', 'de', 'hi', 'ar'],
});
LocalizationLoaded copyWith({
Locale? locale,
TranslationEntity? translations,
List<String>? supportedLocales,
}) {
return LocalizationLoaded(
locale: locale ?? this.locale,
translations: translations ?? this.translations,
supportedLocales: supportedLocales ?? this.supportedLocales,
);
}
@override
List<Object?> get props => [locale, translations, supportedLocales];
}
final class LocalizationError extends LocalizationState {
final String message;
final String? code;
final TranslationEntity? fallbackTranslations;
final Locale? fallbackLocale;
const LocalizationError({
required this.message,
this.code,
this.fallbackTranslations,
this.fallbackLocale,
});
@override
List<Object?> get props => [message, code, fallbackTranslations, fallbackLocale];
}
final class LocaleChanging extends LocalizationState {
final Locale fromLocale;
final Locale toLocale;
const LocaleChanging({
required this.fromLocale,
required this.toLocale,
});
@override
List<Object?> get props => [fromLocale, toLocale];
}
The BLoC handles all localization logic and coordinates between use cases:
/// lib/features/localization/presentation/bloc/localization_bloc.dart
class LocalizationBloc extends Bloc<LocalizationEvent, LocalizationState> {
final GetTranslationsUseCase _getTranslationsUseCase;
final ChangeLocaleUseCase _changeLocaleUseCase;
final GetSupportedLocalesUseCase _getSupportedLocalesUseCase;
final LocalizationRepository _repository;
LocalizationBloc({
required GetTranslationsUseCase getTranslationsUseCase,
required ChangeLocaleUseCase changeLocaleUseCase,
required GetSupportedLocalesUseCase getSupportedLocalesUseCase,
required LocalizationRepository repository,
}) : _getTranslationsUseCase = getTranslationsUseCase,
_changeLocaleUseCase = changeLocaleUseCase,
_getSupportedLocalesUseCase = getSupportedLocalesUseCase,
_repository = repository,
super(const LocalizationInitial()) {
on<InitializeLocalizationEvent>(_onInitialize);
on<LoadTranslationsEvent>(_onLoadTranslations);
on<ChangeLocaleEvent>(_onChangeLocale);
on<RefreshTranslationsEvent>(_onRefreshTranslations);
}
Future<void> _onInitialize(
InitializeLocalizationEvent event,
Emitter<LocalizationState> emit,
) async {
emit(const LocalizationLoading(message: 'Initializing...'));
final savedLocale = await _repository.getPreferredLocale();
final localeToLoad = savedLocale ?? 'en';
final result = await _getTranslationsUseCase(
GetTranslationsParams(locale: localeToLoad),
);
result.fold(
(failure) {
emit(LocalizationError(
message: failure.message,
code: failure.code,
));
},
(translations) {
AppStrings.updateTranslations(
translations.translations.map((k, v) => MapEntry(k, v.toString())),
translations.locale,
);
emit(LocalizationLoaded(
locale: Locale(localeToLoad),
translations: translations,
));
},
);
}
Future<void> _onChangeLocale(
ChangeLocaleEvent event,
Emitter<LocalizationState> emit,
) async {
final currentState = state;
final currentLocale = currentState is LocalizationLoaded
? currentState.locale
: const Locale('en');
if (currentLocale.languageCode == event.locale) return;
emit(LocaleChanging(
fromLocale: currentLocale,
toLocale: Locale(event.locale),
));
final result = await _changeLocaleUseCase(
ChangeLocaleParams(locale: event.locale),
);
result.fold(
(failure) {
if (currentState is LocalizationLoaded) {
emit(currentState);
}
emit(LocalizationError(
message: failure.message,
code: failure.code,
fallbackLocale: currentLocale,
));
},
(translations) {
AppStrings.updateTranslations(
translations.translations.map((k, v) => MapEntry(k, v.toString())),
translations.locale,
);
final supportedLocales = currentState is LocalizationLoaded
? currentState.supportedLocales
: const ['en', 'es', 'fr', 'de', 'hi', 'ar'];
emit(LocalizationLoaded(
locale: Locale(event.locale),
translations: translations,
supportedLocales: supportedLocales,
));
},
);
}
}
The injection container wires everything together:
/// lib/core/di/injection_container.dart
final GetIt sl = GetIt.instance;
Future<void> initDependencies() async {
await Hive.initFlutter();
final sharedPreferences = await SharedPreferences.getInstance();
sl.registerLazySingleton<SharedPreferences>(() => sharedPreferences);
sl.registerLazySingleton<Connectivity>(() => Connectivity());
sl.registerLazySingleton<Dio>(() => _createDio());
sl.registerLazySingleton<NetworkInfo>(
() => NetworkInfoImpl(sl<Connectivity>()),
);
sl.registerLazySingleton<LocalizationRemoteDataSource>(
() => LocalizationRemoteDataSourceImpl(sl<Dio>()),
);
sl.registerLazySingleton<LocalizationLocalDataSource>(
() => LocalizationLocalDataSourceImpl(sl<SharedPreferences>()),
);
sl.registerLazySingleton<LocalizationRepository>(
() => LocalizationRepositoryImpl(
remoteDataSource: sl<LocalizationRemoteDataSource>(),
localDataSource: sl<LocalizationLocalDataSource>(),
networkInfo: sl<NetworkInfo>(),
),
);
sl.registerLazySingleton<GetTranslationsUseCase>(
() => GetTranslationsUseCase(sl<LocalizationRepository>()),
);
sl.registerLazySingleton<ChangeLocaleUseCase>(
() => ChangeLocaleUseCase(sl<LocalizationRepository>()),
);
sl.registerLazySingleton<GetSupportedLocalesUseCase>(
() => GetSupportedLocalesUseCase(sl<LocalizationRepository>()),
);
sl.registerLazySingleton<LocalizationBloc>(
() => LocalizationBloc(
getTranslationsUseCase: sl<GetTranslationsUseCase>(),
changeLocaleUseCase: sl<ChangeLocaleUseCase>(),
getSupportedLocalesUseCase: sl<GetSupportedLocalesUseCase>(),
repository: sl<LocalizationRepository>(),
),
);
}
Dio _createDio() {
final dio = Dio(
BaseOptions(
baseUrl: ApiConfig.baseUrl,
connectTimeout: ApiConfig.connectTimeout,
receiveTimeout: ApiConfig.receiveTimeout,
sendTimeout: ApiConfig.sendTimeout,
headers: ApiConfig.defaultHeaders,
),
);
dio.interceptors.add(
LogInterceptor(
request: true,
requestHeader: true,
requestBody: true,
responseHeader: true,
responseBody: true,
error: true,
),
);
return dio;
}
The backend runs on Node.js with Express and MongoDB Atlas. Here's how to set it up:
The API exposes three endpoints:
GET /api/v1/translations/:locale - Returns translations for a specific localeGET /api/v1/translations/supported-locales - Returns list of available localesGET /api/v1/translations/:locale?current_version=X - Conditional request that returns 304 if version matchesThe translation documents in MongoDB follow this schema:
{
locale: "en",
version: "1.0.0",
updated_at: ISODate("2024-01-15T10:30:00Z"),
translations: {
"app_title": "Localization Demo",
"welcome_message": "Welcome, {name}!",
"login": "Login",
// ... more keys
}
}
When the client sends current_version as a query parameter, the server compares it with the database version. If they match, it returns HTTP 304 with no body. Otherwise, it returns the full translation document with HTTP 200.
Initialize dependencies in main.dart:
/// lib/main.dart
void main() async {
WidgetsFlutterBinding.ensureInitialized();
await initDependencies();
runApp(const LocalizationApp());
}
class LocalizationApp extends StatelessWidget {
const LocalizationApp({super.key});
@override
Widget build(BuildContext context) {
return BlocProvider<LocalizationBloc>(
create: (_) => sl<LocalizationBloc>()
..add(const InitializeLocalizationEvent()),
child: BlocBuilder<LocalizationBloc, LocalizationState>(
buildWhen: (previous, current) {
if (previous is LocalizationLoaded && current is LocalizationLoaded) {
return previous.locale != current.locale;
}
return true;
},
builder: (context, state) {
Locale currentLocale = const Locale('en', 'US');
if (state is LocalizationLoaded) {
currentLocale = state.locale;
}
final isRtl = SupportedLocales.findByCode(
currentLocale.languageCode,
).isRtl;
return MaterialApp(
title: AppStrings.getValue(AppStrings.appTitle),
debugShowCheckedModeBanner: false,
theme: AppTheme.light,
locale: currentLocale,
supportedLocales: SupportedLocales.flutterLocales,
localizationsDelegates: const [
GlobalMaterialLocalizations.delegate,
GlobalWidgetsLocalizations.delegate,
GlobalCupertinoLocalizations.delegate,
],
home: Directionality(
textDirection: isRtl ? TextDirection.rtl : TextDirection.ltr,
child: const HomePage(),
),
);
},
),
);
}
}
Server connection refused on Android emulator:
Use 10.0.2.2 instead of localhost in your base URL. The emulator's localhost refers to itself, not your development machine.
HTTP 304 treated as an error:
Configure Dio's validateStatus to accept 304: validateStatus: (status) => status != null && (status < 400 || status == 304)
Translations not updating:
Check the version number in your MongoDB document. The server only returns 304 if versions match. Increment the version to force a refresh.
Cleartext traffic blocked:
Add android:usesCleartextTraffic="true" to your AndroidManifest.xml during development. Remove it before production.
MongoDB Atlas connection issues:
Ensure your IP address is whitelisted in Atlas Network Access settings. The error "ECONNREFUSED" usually means network restrictions.
Understanding how data flows through the application helps you debug issues and extend the system. Here's the complete journey of a translation request:
┌─────────────────────────────────────────────────────────────────────────────┐
│ APPLICATION STARTUP │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 1. main.dart initializes dependencies via GetIt │
│ └── Registers: Dio, SharedPreferences, Hive, BLoC, UseCases │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 2. LocalizationBloc receives InitializeLocalizationEvent │
│ └── Checks SharedPreferences for saved locale preference │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 3. GetTranslationsUseCase calls LocalizationRepository │
│ └── Repository checks network connectivity │
└─────────────────────────────────────────────────────────────────────────────┘
│
┌─────────────────┴─────────────────┐
│ │
[ONLINE] [OFFLINE]
│ │
▼ ▼
┌───────────────────────────────┐ ┌───────────────────────────────────────┐
│ 4a. RemoteDataSource calls │ │ 4b. LocalDataSource reads from │
│ backend API via Dio │ │ Hive cache │
│ GET /translations/{locale}│ │ │
│ ?current_version=X │ │ │
└───────────────────────────────┘ └───────────────────────────────────────┘
│ │
┌─────────┴─────────┐ │
│ │ │
[HTTP 200] [HTTP 304] │
│ │ │
▼ ▼ │
┌─────────────────┐ ┌─────────────────┐ │
│ New translations│ │ CacheValidExcep │ │
│ returned │ │ tion thrown │ │
└─────────────────┘ └─────────────────┘ │
│ │ │
▼ └────────────┬────────────┘
┌─────────────────┐ │
│ Cache new data │ │
│ in Hive │ │
└─────────────────┘ │
│ │
└───────────────┬────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 5. Repository returns Either<Failure, TranslationEntity> │
│ └── BLoC emits LocalizationLoaded state with translations │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 6. AppStrings.updateTranslations() stores translations in memory │
│ └── UI rebuilds with BlocBuilder, displaying localized content │
└─────────────────────────────────────────────────────────────────────────────┘
When a user selects a new language:
User taps language ──► ChangeLocaleEvent dispatched
│
▼
BLoC emits LocaleChanging state (shows loading indicator)
│
▼
ChangeLocaleUseCase fetches new translations
│
▼
On success: Save preference + Cache translations + Emit LocalizationLoaded
│
▼
MaterialApp rebuilds with new locale
│
▼
Directionality widget updates for RTL languages (Arabic)
The HTTP 304 mechanism saves bandwidth and improves performance:
First Request (no cache):
Client: GET /translations/en
Server: 200 OK + Full translation data (version: 1.0.0)
Client: Stores in Hive with version
Subsequent Request (with cache):
Client: GET /translations/en?current_version=1.0.0
Server: Compares versions
└── Same version → 304 Not Modified (no body)
└── New version → 200 OK + Updated translations
Result: Only downloads data when translations actually change
Building on this foundation, here are strategies to maximize performance:
Lazy Loading Translations
Instead of loading all translations at startup, consider loading only the current locale and prefetching others in the background:
// In your BLoC, after loading primary locale
Future<void> _prefetchOtherLocales() async {
final otherLocales = ['es', 'fr', 'de'].where((l) => l != currentLocale);
for (final locale in otherLocales) {
await _repository.getTranslations(locale); // Caches silently
}
}
Memory-Efficient String Access
The AppStrings.getValue() method avoids creating new string objects for cached values:
// Efficient: Retrieves from in-memory map
Text(AppStrings.getValue(AppStrings.login))
// Less efficient: Creates interpolated string each time
Text('${translations['login']}')
Minimize Rebuilds
Use buildWhen in BlocBuilder to prevent unnecessary widget rebuilds:
BlocBuilder<LocalizationBloc, LocalizationState>(
buildWhen: (previous, current) {
// Only rebuild when locale actually changes
if (previous is LocalizationLoaded && current is LocalizationLoaded) {
return previous.locale != current.locale;
}
return true;
},
builder: (context, state) => /* your widget */,
)
A robust localization system needs thorough testing. Here are the key test scenarios:
Unit Tests for Repository
test('should return cached translations when server returns 304', () async {
// Arrange
when(mockRemoteDataSource.getTranslations('en', currentVersion: '1.0.0'))
.thenThrow(const CacheValidException());
when(mockLocalDataSource.getCachedTranslations('en'))
.thenAnswer((_) async => tTranslationModel);
// Act
final result = await repository.getTranslations('en');
// Assert
expect(result, Right(tTranslationModel));
verify(mockLocalDataSource.getCachedTranslations('en'));
});
Integration Test for Language Switching
testWidgets('should switch language and update UI', (tester) async {
await tester.pumpWidget(const LocalizationApp());
await tester.pumpAndSettle();
// Verify initial English text
expect(find.text('Login'), findsOneWidget);
// Navigate to settings and change language
await tester.tap(find.byIcon(Icons.settings));
await tester.pumpAndSettle();
await tester.tap(find.text('Español'));
await tester.pumpAndSettle();
// Verify Spanish text appears
expect(find.text('Iniciar sesión'), findsOneWidget);
});
Offline Mode Test
test('should use cached translations when offline', () async {
// Arrange
when(mockNetworkInfo.isConnected).thenAnswer((_) async => false);
when(mockLocalDataSource.getCachedTranslations('en'))
.thenAnswer((_) async => tTranslationModel);
// Act
final result = await repository.getTranslations('en');
// Assert
expect(result.isRight(), true);
verifyNever(mockRemoteDataSource.getTranslations(any));
});
When implementing backend-driven localization in production:
API Security
usesCleartextTraffic from AndroidManifest)Content Validation
Cache Security
Building a backend-driven localization system requires thoughtful architecture, but the benefits far outweigh the initial complexity. You now have a system where translations live independently from your app binary, updates happen instantly without app store delays, and users always see the most current content regardless of their app version.
The Clean Architecture approach pays dividends as your app grows. Need to switch from Hive to SQLite for caching? Change only the local data source. Want to add WebSocket support for real-time updates? Modify the remote data source and repository. The domain layer with its use cases and entities remains untouched, protecting your business logic from infrastructure changes.
The HTTP 304 mechanism might seem like a small optimization, but at scale it significantly reduces bandwidth costs and improves app responsiveness. Users with slow connections appreciate faster language switches when translations are already cached.
Consider this implementation a starting point. The patterns established here extend naturally to other dynamic content: feature flags, remote configuration, A/B testing variants, and more. Once you have the infrastructure for fetching, caching, and displaying dynamic content, applying it to new use cases becomes straightforward.
The combination of offline-first caching, intelligent cache validation, and robust error handling creates an experience that feels native while providing the flexibility of server-driven content. Users get instant language switching with cached data, automatic updates when new translations are available, and graceful fallbacks when network conditions are poor.
The complete implementation is available on GitHub. Clone the repository, configure your backend URL, and you'll have a working localization system ready for customization.
GitHub Repository: https://github.com/Anurag-Dubey12/backend-based-Localization/tree/main
The repository includes:
2025-12-21 00:25:32
I ran a rigorous benchmark comparing the two major ESLint security plugins. This article covers the full methodology, test files, and results—including why 0 false positives matters more than raw speed.
vulnerable.js (218 lines) - Contains 12 categories of real vulnerabilities:
// 1. Command Injection
exec(`ls -la ${userInput}`);
execSync('echo ' + userInput);
spawn('bash', ['-c', userInput]);
// 2. Path Traversal
fs.readFile(filename, 'utf8', callback);
fs.readFileSync(filename);
// 3. Object Injection
obj[key] = value;
data[key][value] = 'test';
// 4. SQL Injection
db.query('SELECT * FROM users WHERE id = ' + userId);
// 5. Code Execution
eval(code);
new Function(code);
// 6. Regex DoS
const evilRegex = /^(a+)+$/;
new RegExp(userInput);
// 7. Weak Cryptography
crypto.createHash('md5').update(password);
Math.random().toString(36);
// 8. Timing Attacks
if (inputToken === storedToken) {
return true;
}
// 9. XSS
document.getElementById('output').innerHTML = userContent;
// 10. Insecure Cookies
document.cookie = `${name}=${value}`;
// 11. Dynamic Require
require(moduleName);
// 12. Buffer Issues
const buf = new Buffer(size);
safe-patterns.js (167 lines) - Contains defensive patterns that should NOT trigger warnings:
// Safe: Validated key access with allowlist
const VALID_KEYS = ['name', 'email', 'age'];
if (VALID_KEYS.includes(key)) {
return obj[key];
}
// Safe: hasOwnProperty check
if (Object.prototype.hasOwnProperty.call(obj, key)) {
return obj[key];
}
// Safe: Path validation with startsWith
if (!safePath.startsWith(SAFE_DIR)) throw new Error('Invalid');
fs.readFileSync(safePath);
// Safe: Timing-safe comparison
crypto.timingSafeEqual(bufA, bufB);
// Safe: DOMPurify sanitization
const clean = DOMPurify.sanitize(userContent);
element.innerHTML = clean;
First, I tested both plugins with only the 14 equivalent rules that exist in both packages. This ensures an apples-to-apples comparison.
| Metric | secure-coding |
security |
Winner |
|---|---|---|---|
| Performance/Issue | 24.95ms | 25.12ms | 🟢 secure-coding |
| Total Time | 723.54ms | 527.58ms | 🔵 security |
| Issues Found | 29 | 21 | 🟢 secure-coding |
| Detection Rate | 138% | 100% | 🟢 secure-coding |
| Rule Category | security |
secure-coding |
Diff |
|---|---|---|---|
| Timing Attacks | 1 | 5 | +4 🟢 |
| Child Process | 2 | 4 | +2 🟢 |
| Non-literal Regexp | 1 | 3 | +2 🟢 |
| Eval/Code Execution | 1 | 2 | +1 🟢 |
| Insufficient Randomness | 0 | 1 | +1 🟢 |
| FS Path Traversal | 5 | 5 | = |
| Object Injection | 5 | 5 | = |
| Dynamic Require | 2 | 2 | = |
| Unsafe Regex | 2 | 2 | = |
| Buffer APIs | 2 | 0 | -2 🔵 |
| TOTAL | 21 | 29 | +8 |
Key Finding: With the same rule categories, secure-coding finds 38% more issues while maintaining nearly identical efficiency per issue.
Next, I tested each plugin's recommended configuration—the out-of-box experience.
| Metric | secure-coding |
security |
Winner |
|---|---|---|---|
| Performance/Issue | 9.95ms | 28.16ms | 🟢 secure-coding |
| Total Time | 795.99ms | 591.41ms | 🔵 security |
| Issues Found | 80 | 21 | 🟢 secure-coding |
| Rules Triggered | 30 | 10 | 🟢 secure-coding |
| Total Rules | 89 | 14 | 🟢 secure-coding |
secure-coding rules triggered on vulnerable.js:
• no-unvalidated-user-input: 8 issues
• detect-non-literal-fs-filename: 5 issues
• detect-object-injection: 5 issues
• no-timing-attack: 5 issues
• detect-child-process: 4 issues
• database-injection: 4 issues
• no-unsafe-deserialization: 4 issues
• no-sql-injection: 3 issues
• detect-non-literal-regexp: 3 issues
• no-hardcoded-credentials: 2 issues
• detect-eval-with-expression: 2 issues
• no-weak-crypto: 2 issues
... and 18 more categories
security rules triggered:
• detect-non-literal-fs-filename: 5 issues
• detect-object-injection: 5 issues
• detect-child-process: 2 issues
• detect-unsafe-regex: 2 issues
... and 6 more categories
This is where precision matters. I ran both plugins against safe-patterns.js—a file with only safe, validated code.
| Plugin | False Positives | Precision |
|---|---|---|
secure-coding |
0 | 100% |
security |
4 | 84% |
eslint-plugin-security
FP #1: Validated key access (line 38)
// Pattern: Allowlist validation before access
const VALID_KEYS = ['name', 'email', 'age'];
function getField(obj, key) {
if (VALID_KEYS.includes(key)) {
return obj[key]; // ⚠️ security flags "Generic Object Injection Sink"
}
}
The developer validated key against an allowlist. This is a safe pattern.
FP #2: hasOwnProperty check (line 45)
// Pattern: Property existence check before access
function safeGet(obj, key) {
if (Object.prototype.hasOwnProperty.call(obj, key)) {
return obj[key]; // ⚠️ security flags "Generic Object Injection Sink"
}
}
hasOwnProperty ensures key exists on the object itself, not the prototype chain.
FP #3: Guard clause with throw (line 153)
// Pattern: Early exit guard clause
const ALLOWED_THEMES = ['light', 'dark', 'system'];
function setTheme(userTheme) {
if (!ALLOWED_THEMES.includes(userTheme)) {
throw new Error('Invalid theme');
}
config[userTheme] = true; // ⚠️ security flags despite throw guard
}
The throw acts as a guard—execution cannot reach line 153 with an invalid theme.
FP #4: Path validation (line 107)
// Pattern: basename + startsWith validation
function safeReadFile(userFilename) {
const safeName = path.basename(userFilename);
const safePath = path.join(SAFE_DIR, safeName);
if (!safePath.startsWith(SAFE_DIR)) {
throw new Error('Invalid path');
}
return fs.readFileSync(safePath); // ⚠️ security flags "non literal argument"
}
The path is fully validated: basename strips traversal, startsWith confirms the directory.
secure-coding Avoids These
We use AST-based validation detection:
| Pattern | Detection Method |
|---|---|
allowlist.includes(key) |
Check for includes() in enclosing if-statement |
hasOwnProperty(key) |
Check for hasOwnProperty/hasOwn call |
| Guard clause + throw | Detect preceding IfStatement with early exit |
startsWith() validation |
Detect path validation patterns |
| Coverage | secure-coding |
security |
|---|---|---|
| OWASP Web Top 10 | 10/10 (100%) | ~3/10 (~30%) |
| OWASP Mobile Top 10 | 10/10 (100%) | 0/10 (0%) |
| Total | 20/20 | ~3/20 |
Security rules are increasingly consumed by AI coding assistants. Compare the messages:
eslint-plugin-security:
Found child_process.exec() with non Literal first argument
eslint-plugin-secure-coding:
🔒 CWE-78 OWASP:A03-Injection CVSS:9.8 | Command injection detected | CRITICAL
Fix: Use execFile/spawn with {shell: false} and array args
📚 https://owasp.org/www-community/attacks/Command_Injection
| Feature | secure-coding |
security |
|---|---|---|
| CWE ID | ✅ | ❌ |
| OWASP Category | ✅ | ❌ |
| CVSS Score | ✅ | ❌ |
| Fix Instructions | ✅ | ❌ |
| Documentation Link | ✅ | ❌ |
Beyond detection metrics, here's the full feature comparison:
| Feature | secure-coding |
security |
|---|---|---|
| Total Rules | 89 | 14 |
| Documentation | Comprehensive (per-rule) | Basic |
| Fix Suggestions/Rule | 3-6 suggestions | 0 |
| CWE References | ✅ All rules | ❌ None |
| CVSS Scores | ✅ Yes | ❌ No |
| OWASP Mapping | ✅ Web + Mobile | ❌ None |
| TypeScript Support | ✅ Full | ⚠️ Partial |
| Flat Config Support | ✅ Native | ✅ Native |
| Presets | minimal, recommended, strict | recommended |
| Last Updated | Active | Maintenance mode |
| Category | secure-coding |
security |
Winner |
|---|---|---|---|
| Performance/Issue | 9.95ms | 28.16ms | 🟢 secure-coding |
| Detection | 80 issues | 21 issues | 🟢 secure-coding |
| False Positives | 0 | 4 | 🟢 secure-coding |
| Precision | 100% | 84% | 🟢 secure-coding |
| Total Rules | 89 | 14 | 🟢 secure-coding |
| OWASP Coverage | 20/20 | ~3/20 | 🟢 secure-coding |
| Documentation | Comprehensive | Basic | 🟢 secure-coding |
| Fix Suggestions | 3-6 per rule | 0 | 🟢 secure-coding |
| LLM Optimization | ⭐⭐⭐⭐⭐ | ⭐⭐ | 🟢 secure-coding |
Performance per issue matters — secure-coding is 2.83x more efficient per detected issue.
"Speed advantage" = detection gap — The incumbent is faster because it misses vulnerabilities.
0 false positives — Every flagged issue is a real vulnerability.
6x more rules — 89 rules vs 14, covering web, mobile, API, and AI security.
Developer experience — Every rule includes CWE/OWASP references, CVSS scores, and 3-6 fix suggestions.
npm install eslint-plugin-secure-coding --save-dev
// eslint.config.js
import secureCoding from 'eslint-plugin-secure-coding';
export default [secureCoding.configs.recommended];
The benchmark code is open source: benchmark on GitHub
2025-12-21 00:25:10
I am a B.Tech Computer Science undergraduate at Amrita Vishwa Vidyapeetham who enjoys building privacy-focused systems and learning by deploying real software end-to-end.
The inception of InfoStuffs was not driven by a desire to build just another productivity application. It started with a very specific user requirement from my sister. She needed a digital space to organize personal documents and sensitive notes but refused to use standard cloud services such as Google Keep or Notion.
Her constraint was simple but technically demanding: she wanted the convenience of the cloud without trusting the cloud provider with her plaintext data.
This challenge became the foundation of InfoStuffs. My goal shifted from building a simple web application to architecting a Zero-Trust Information Management System that prioritizes privacy by default.
Modern productivity tools generally fall into two categories:
InfoStuffs bridges this gap. The system had to be secure enough that a complete server-side compromise would yield nothing but garbage data, yet accessible via a standard web browser on any device.
To satisfy these constraints, InfoStuffs uses a decoupled, cloud-native architecture with clearly separated responsibilities.
Backend: Node.js and Express, following a Hybrid Architecture
Database: MongoDB Atlas for storing encrypted metadata and ciphertext.
Authentication: Clerk. Delegating identity management reduced the attack surface for auth flows (MFA, session management).
Storage: Supabase Storage, used strictly for isolating binary objects (images and PDFs) via signed URLs.
Security was not an optional feature; it was the primary architectural constraint.
In my initial design, I used a static encryption key stored in the server's environment variables (VITE_SECRET_KEY). I quickly realized this was a critical flaw. If an attacker or a compromised hosting environment were to expose environment variables, they could decrypt everyone's data. The key was visible, which violated the core concept of Zero-Trust.
To fix this, I removed the static key entirely. I implemented PBKDF2 (Password-Based Key Derivation Function 2) on the client side.
The server only ever sees (and stores) ciphertext. If the database administrator (me) were to look at the data, I would see nothing but unreadable strings.
The PBKDF2 parameters were chosen to balance resistance to brute-force attacks with acceptable latency on low-powered client devices.
For file storage, I avoided public buckets entirely.
One of the most valuable learning experiences came from adapting the infrastructure to real-world cost constraints.
My initial deployment used Google Cloud Platform with Cloud Run and Cloud Build. While this was an industry-standard "Enterprise" setup, it introduced significant problems for a personal project:
To solve the deployment cost problem, I re-architected the stack to run for $0/month:
docker-compose up spins up the Frontend, Backend, and Database services. This ensures that the development environment is isolated and reproducible on any machine.I intentionally avoided microservices, as the domain does not yet justify multiple bounded contexts, and premature service decomposition would increase complexity and attack surface without tangible benefits.
1. Environment Variable Visibility
As mentioned, relying on .env files for security was a mistake. The migration to user-derived keys solved this, but it required handling edge cases like "Lost Passwords." Since I no longer had the key, I had to implement a "Nuclear Reset" feature. This allows users to wipe their unrecoverable data and start fresh, prioritizing security over recovery.
2. Monorepo Build Contexts
Vercel initially failed to build the project because it couldn't locate vite.config.js within the monorepo structure. I resolved this by explicitly configuring the "Root Directory" in Vercel settings and rewriting the build command to ensure dependencies were installed from the correct path.
3. The Docker vs. Serverless Routing Mismatch
One of the most complex challenges was reconciling the difference between a running Docker container and Vercel's file-system routing.
The Problem: In my local Docker container, Express handled all routing internally. However, when deployed to Vercel, the platform treated the API as static files. Requests to sub-paths (like /api/info/nuke) were hitting Vercel's 404 handler before reaching my Express app, which the browser misinterpreted as a CORS failure.
The Solution: I implemented a Hybrid Routing Strategy. I created a Vercel-compatible entry point (api/info.js) and configured a vercel.json rewrite rule. This acts as a bridge, telling Vercel to pipe all sub-route traffic directly into the Express instance. This fixed the CORS issues and allowed the exact same code to run inside Docker (locally) and as a Function (in production).
While InfoStuffs is fully functional, I plan to explore:
InfoStuffs is more than a note-taking application. It is a practical exploration of Zero-Trust Engineering.
By addressing the real-world problems of data visibility and cloud costs, I built a system where privacy is enforced by mathematics, not by policy. It satisfies a real user need while serving as a valuable learning experience in full-stack security and DevOps.
Repository: GitHub
Live Deployment: Link
Note: The live deployment requires authentication and a vault password. The core security properties are enforced client-side and are best understood via the architecture discussion above.
2025-12-21 00:24:48
The Terraform language includes a number of built-in functions that you can call from within expressions to transform and combine values. The general syntax for function calls is a function name followed by comma-separated arguments in parentheses.
lower converts all cased letters in the given string to lowercase.
Examples
> lower("HELLO")
hello
This function uses Unicode's definition of letters and of upper- and lowercase.
replace searches a given string for another given substring and replaces each occurrence with a given replacement string.
replace(string, substring, replacement)
Examples
> replace("1 + 2 + 3", "+", "-")
1 - 2 - 3
> replace("hello world", "/w.*d/", "everybody")
hello everybody
substr extracts a substring from a given string by offset and (maximum) length.
substr(string, offset, length)
Examples
> substr("hello world", 1, 4)
ello
split produces a list by dividing a given string at all occurrences of a given separator.
split(separator, string)
Examples
> split(",", "foo,bar,baz")
[
"foo",
"bar",
"baz",
]
> split(",", "foo")
[
"foo",
]
> split(",", "")
[
"",
]
merge takes an arbitrary number of maps or objects, and returns a single map or object that contains a merged set of elements from all arguments.
If more than one given map or object defines the same key or attribute, then the one that is later in the argument sequence takes precedence. If the argument types do not match, the resulting type will be an object matching the type structure of the attributes after the merging rules have been applied.
Examples
> merge({a="b", c="d"}, {e="f", c="z"})
{
"a" = "b"
"c" = "z"
"e" = "f"
}
> merge({a="b"}, {a=[1,2], c="z"}, {d=3})
{
"a" = [
1,
2,
]
"c" = "z"
"d" = 3
}
lookup retrieves the value of a single element from a map, given its key. if the given key does not exist, the given default value is returned instead.
lookup(map, key, default)
Examples
> lookup({a="ay", b="bee"}, "a", "what?")
ay
> lookup({a="ay", b="bee"}, "c", "what?")
what?
If you enjoyed this post or want to follow my #30DaysOfAWSTerraformChallenge journey, feel free to connect with me here:
💼 LinkedIn: Amit Kushwaha
🐙 GitHub: Amit Kushwaha
📝 Hashnode / Amit Kushwaha
🐦 Twitter/X: Amit Kushwaha
2025-12-21 00:22:34
As cloud-native systems continue to grow in complexity, many organizations depend on Kubernetes to run and scale their containerized applications. While Kubernetes is powerful, managing distributed workloads often comes with limited visibility, making it difficult to detect issues before they impact the business. The real challenge isn’t just deploying applications to a Kubernetes cluster—it’s understanding how those applications are performing, how resources are being used, and whether the system is healthy overall. In today's blog we look at how building an observability-first Amazon Elastic Kubernetes Service (EKS) platform can help solve these challenges through better monitoring, automated scaling, and early detection of potential incidents.
This project aims to demonstrates how we can design, provision, and operate a production‑style, observability‑first Kubernetes platform on Amazon EKS, using Terraform as the platform definition layer.
The focus is Day‑2 operations: metrics, autoscaling, failure recovery, and clean platform boundaries — not just deploying containers.
.
├── providers.tf
├── versions.tf
├── variables.tf
├── main.tf
├── outputs.tf
├── modules/
│ ├── vpc/
│ ├── eks/
│ ├── observability/
│ └── apps/
Each Terraform module represents a platform responsibility boundary.
EKS Module: Deploys managed Kubernetes cluster (v1.32) with:
Application Stack
containous/whoami service showing request detailshashicorp/http-echo returning "Hello from API"Why: EKS must run in private subnets for production‑grade security.
Key takeaway: 🌟 Networking is platform infrastructure, not app concern. 🌟
Why: Managed control plane + managed node groups reduce operational load.
Key takeaway: 🌟 Platform teams optimize for operability, not customization. 🌟
Why: You cannot safely scale or debug what you cannot see.
observability namespacekube‑prometheus‑stack via HelmKey takeaway: 🌟 Observability is foundational infrastructure. 🌟
Why: Namespace isolation simplifies ownership and RBAC later.
apps namespaceKey takeaway: 🌟 Logical isolation improves long‑term operability. 🌟
Why: Complete the 3‑tier story, even with a simple UI.
Key takeaway: 🌟 Even simple workloads deserve resource boundaries. 🌟
Why: This tier demonstrates autoscaling and failure recovery.
http‑echo)Key takeaway: 🌟 Backend services are the primary scaling surface. 🌟
Why: Scaling without metrics is dangerous.
Key takeaway: 🌟 Autoscaling is a control system, not a checkbox. 🌟
Key takeaway: 🌟 Resilience is observable, not assumed. 🌟
terraform init
terraform validate
terraform plan
terraform apply

Explore a bit to identify the resources created, networking layer etc.
Check current frontend service: kubectl get svc -n apps
If no service exists, create one:kubectl patch svc frontend -n apps -p '{"spec": {"type": "LoadBalancer"}}'kubectl patch svc api -n apps -p '{"spec": {"type": "LoadBalancer"}}'
Get the external URL:kubectl get svc frontend -n appskubectl get svc api -n apps
Wait for EXTERNAL-IP to show the AWS ELB hostname (takes 2-3 minutes).
Get the hostname directly:kubectl get svc frontend -n apps -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
kubectl get svc api -n apps -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
Access your frontend
Once you have the hostname, access it in your browser:http://your-elb-hostname
kubectl port-forward -n observability svc/kube-prometheus-grafana 3000:80
Open browser:
http://localhost:3000
Login:
Decode the Grafana Configure and View dashboards! Follow same approach for Prometheus: Dashboards to open: Confirm: Exec into API pod: Generate CPU load: Observe: Observe: List nodes: Drain one node: Observe: A complete Terraform-based solution for deploying an Amazon EKS cluster with built-in observability stack and sample applications. This project provisions a production-ready EKS cluster on AWS with:admin password: $password = kubectl get secret -n observability kube-prometheus-grafana -o jsonpath="{.data.admin-password}"
Observe Baseline Metrics
Generate Load (Autoscaling Demo)
kubectl exec -it deploy/api -n apps -- shwhile true; do :; done
Pod Failure Injection
kubectl delete pod -n apps -l app=api
Node Failure Injection
kubectl get nodeskubectl drain <node-name> --ignore-daemonsets
Github Repository
aggarwal-tanushree
/
eks-observability-first-platform
Personal platform engineering project demonstrating an observability-first 3-tier architecture on Amazon EKS using Terraform, Prometheus, and Grafana.
EKS Observability First Platform
Description
Directory Structure
…eks-observability-first-platform/
├── modules/
│ ├── vpc/ # VPC module
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ ├── eks/ # EKS cluster module
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ ├── observability/ # Prometheus/Grafana stack
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── versions.tf
│ └── apps/ # Sample applications
│ ├── main.tf
│ ├── variables.tf
│ └── versions.tf
├── main.tf # Root module
Why: Over‑provisioning nodes is the most common EKS cost mistake.
Frontend: Requests : 50m CPU / 64Mi memory. Limits : 200m CPU / 128Mi memory
Backend API: Requests : 100m CPU / 128Mi memory. Limits : 500m CPU / 256Mi memory
Why:
Cost Benefit:
Metrics help avoid:
Key Insight:
Observability is a cost‑control mechanism, not just a debugging tool.
This demo proves that EKS platforms must be observable before they are scalable.
The Operational Challenge:
As organizations adopt Kubernetes, many run into an unexpected contradiction. While containers and orchestration make it easier to scale and move faster, they also add complexity that makes systems harder to understand. As a result, teams often end up reacting to problems after something breaks instead of preventing issues through better visibility and automation.
The Observability-First Approach
The answer is to adopt an observability-first approach—one where monitoring and visibility are built into the platform from day one, not added later as an afterthought. When teams can clearly see what’s happening inside their systems, they’re able to spot issues early, make smarter decisions automatically, and continuously improve performance. This shift allows organizations to move from constantly reacting to problems to predicting and preventing them.
In an observability-first platform, monitoring is woven directly into the infrastructure as it’s being created. Every component is instrumented and visible as soon as it goes live. This creates a strong foundation for automatic scaling, meaningful alerts, and data-driven capacity planning. Over time, the platform becomes more self-aware—able to understand how it’s performing and adjust on its own as conditions change.
So to summarize, in this (lengthy, but hopefully insightful) blog, we:
✅ Built a production‑style Amazon EKS platform using modular Terraform, separating networking, cluster, observability, and application concerns.
✅ Implemented Prometheus and Grafana before workloads, enabling safe CPU‑based autoscaling and rapid failure detection.
✅ Validated Day‑2 operations by demonstrating pod and node failure recovery with real‑time metrics.
Remember, "If you can’t observe it, you can’t operate it." 🙏
🚀 ALB Ingress + path‑based routing
🚀 An Interactive Web UI for Frontend
🚀 Distributed tracing (OpenTelemetry)
🚀 RBAC per namespace
🚀 CI/CD pipeline integration
🚀 Cost dashboards
https://developer.hashicorp.com/terraform/tutorials/kubernetes/eks
https://registry.terraform.io/providers/hashicorp/aws/latest
2025-12-21 00:07:59
We spend countless hours optimizing LLM prompts, tweaking retrieval parameters (k-NN), and choosing the best embedding models. But we often ignore the elephant in the room: The Data Quality.
If you are building a RAG (Retrieval-Augmented Generation) pipeline using internal company data-logs, tickets, documentation, or emails you have likely encountered the Semantic Duplicate Problem.
The Problem: Different Words, Same Meaning
Standard deduplication tools (like Pandas drop_duplicates() or SQL DISTINCT) work on a string level. They look for exact matches.
Consider these two log entries:
Error: Connection to database timed out after 3000ms.
DB Connection Failure: Timeout limit reached (3s).
To a standard script, these are two unique rows.
To an LLM (and to a human), they are identical.
If you ingest 10,000 such rows into your Vector Database (Pinecone, Milvus, Weaviate):
💸 You waste money: Storing vectors isn't free.
📉 You hurt retrieval: When a user asks "Why did the DB fail?", the retriever fills the context window with 5 variations of the same error, crowding out other relevant information.
😵 Model Hallucinations: The LLM gets repetitive context and degrades in quality.
The Solution: Semantic Deduplication
To fix this, we need to deduplicate based on meaning (vectors), not just syntax (text).
I couldn't find a lightweight tool that does this efficiently on a local machine (Privacy First!) without spinning up a Spark cluster or sending sensitive data to OpenAI APIs. So, I engineered one.
Meet EntropyGuard.
🛡️ EntropyGuard: A Local-First ETL Engine
EntropyGuard is an open-source CLI tool written in Python. It acts as a sanitation layer before your data hits the Vector Database.
It solves three critical problems:
Semantic Deduplication: Uses sentence-transformers and FAISS to find duplicates by cosine similarity.
Sanitization: Strips PII (emails, phones) and HTML noise.
Privacy: Runs 100% locally on CPU. No data exfiltration.
The Tech Stack (Hard Tech)
I wanted this tool to be robust enough for Enterprise data but light enough to run on a laptop.
Engine: Built on Polars LazyFrame. This allows streaming execution. You can process a 10GB CSV on a laptop with 16GB RAM because the data isn't loaded into memory all at once.
Vector Search: Uses FAISS (Facebook AI Similarity Search) for blazing-fast vector comparisons on CPU.
Chunking: Implemented a native recursive chunker (paragraphs -> sentences) to prepare documents for embedding, avoiding the bloat of heavy frameworks like LangChain.
Ingestion: Supports Excel (.xlsx), Parquet, CSV, and JSONL natively.
How it works (The Code)
Using it is as simple as running a CLI command.
Installation:
pip install "git+https://github.com/DamianSiuta/entropyguard.git"
Running the Audit:
You can run a "Dry Run" to generate a JSON audit log showing exactly which rows would be dropped and why. This is crucial for compliance teams.
entropyguard \
--input raw_data.jsonl \
--output clean_data.jsonl \
--dedup-threshold 0.85 \
--audit-log audit_report.json
The Result:
The tool generates embeddings locally (using a small model like all-MiniLM-L6-v2), clusters them using FAISS, and removes neighbors that are closer than the threshold (e.g., 0.85 similarity).
Benchmark: 99.5% Noise Reduction
I ran a stress test using a synthetic dataset of 10,000 rows generated by multiplying 50 unique signals with heavy noise (HTML tags, rephrasing, typos).
Raw Data: 10,000 rows.
Cleaned Data: ~50 rows.
Execution Time: < 2 minutes on a standard laptop CPU.
The tool successfully identified that
System Error and Critical Failure: System were the same event, collapsing them into one canonical entry.Why Open Source?
I believe data hygiene is a fundamental problem that shouldn't require expensive SaaS subscriptions. I released EntropyGuard under the MIT License so anyone can use it—even in commercial/air-gapped environments.
Check out the repo here:
👉 github.com/DamianSiuta/entropyguard
I’m actively looking for feedback from the Data Engineering community. If you are struggling with dirty RAG datasets, give it a spin and let me know if it helps!