Software Engineering Principles

Well - the final draft of the Code of Practice for general-purpose artificial intelligence models is out. On its face, it appears to be a voluntary, self-regulatory initiative encouraging best practices around copyright and AI training. But... The final text, along with new statements from the EU, shows that the Code is closer to a soft mandate. Far from offering clarity, its release highlights how much work is left to be done in Europe’s AI regulatory structure. The Code is far stricter than earlier drafts. Language once referring to “reasonable efforts” has been replaced with binding commitments. Developers must now exclude infringing websites from training data and apply safeguards to stop models from generating outputs that reproduce protected content. They must respect machine-readable rights signals like robots.txt. Crucially, the final text confirms that open-source models are not exempt and OS developers must include documentation warning users that copyright-infringing uses are still prohibited. While the final draft of the Code strengthens obligation wording, it removes the one provision that required developers to assess the legality of third-party datasets. Measure I.2.4 would have obliged them to make basic efforts to verify whether datasets obtained through means other than web crawling were lawfully compiled. Well. It’s gone. This omission creates a large compliance gap. If a signatory scrapes content themselves or commissions someone to do it, obligations apply. But it seems if they purchase a dataset from a third party, there is no obligation to check its origin. In an industry reliant on massive troves of unclear content, that is a notable concession. But wait - there’s more! The Commission’s Q&A confirms that developers who who fall short after signing will still be considered to be acting “in good faith”, and the AI Office will support rather than penalise them. This grace period runs until 2 August 2026, after which fines may be imposed under the AI Act. This sets up a two-tier system. Signatories are shielded for a year, even if non-compliant. Non-signatories are not. So those who do not sign the “voluntary” Code face immediate legal risk. That is not voluntary. It is regulatory pressure disguised as choice. The Code also arrives without critical supports. Although it demands transparency around training data, we still have no template for how disclosures should be made. Article 53 of the AI Act requires developers to publish training data summaries, but no guidelines or minimum content standards have been issued. Nor have the GPAI guidelines been finalised, meaning developers do not yet know how their models will be assessed under EU law. The EU may address these gaps as the AI Act takes effect and the GPAI guidelines are finalised. But for now, this Code is not a definitive rulebook. It is a political compromise, and it leaves critical questions unanswered - especially around data provenance.

Many organizations that invest in Configuration Management do so by heavily investing in digital transformation, but quietly undermine it through inadequate knowledge support and misaligned tools. That’s not a tooling problem. It’s a maturity problem because CM maturity isn’t defined solely by what is defined. It’s about what people understand, can access, and are actually supported by. 👉 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆: If CM knowledge isn’t shared and tools don’t reinforce the process, maturity will remain fragile. 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 & 𝗦𝘂𝗽𝗽𝗼𝗿𝘁. In mature CM organizations, standard CM terminology is documented, validated, released, and accessible to everyone who touches configuration information. There’s no or little tribal knowledge. Training is treated the same way. Not as a one-off rollout, but as a continuous capability: 🔹 Regular CM training across the company 🔹 Targeted, ad-hoc training when changes occur 🔹 Coverage of process, tools, and practical application Mature organizations actively promote access to the latest standards, lessons learned, best practices, and internal and external benchmarks. CM knowledge support is visible, accessible, and trusted, not buried in folders or locked behind specialists. Each improvement becomes the new foundation for future growth. Then comes the topic that often dominates conversations: 𝗧𝗼𝗼𝗹𝘀. Tools don’t create CM maturity. But poor tool decisions can destroy it fast. Mature CM organizations first identify which software capabilities are required to support CM processes: planning, identification, change, status accounting, and verification, before selecting or configuring tools. That maturity shows up when: 🔹 Tool performance is monitored using KPIs 🔹 Strengths and weaknesses are explicitly identified 🔹 Improvement actions are prioritized and captured in a CM roadmap 📍 Effective CM tools are: 🔹 User-friendly and deployed to all relevant users 🔹 Capable of supporting baselining, effectivity, traceability, workflows, and impact analysis 🔹 Able to manage legacy data without breaking traceability 🔹 Integrated where needed with other enterprise tools 🔹 Not rigid or overly configured/customized. They rely on a robust process and guardrails, not hard coded, difficult to maintain complexity that results in a fragile infrastructure. And here’s a detail often overlooked: Those directing software development and upgrades must understand CM. Training, certification, or experience in CM isn’t optional when tools define how configuration management is executed. And if tools can’t support the CM roadmap, or the vendor roadmap isn’t aligned, maturity stalls, no matter how good the intent. 👉 Where does CM maturity break down in your organization: knowledge, support, or tools? 👉 And are your tools enabling CM… or quietly working against it? I’m interested in your experience. #ConfigurationManagement #CM2 #CM #PLM #MaturityAssessment #DigitalTransformation #ProductLifecycleManagement

Don’t break your code during refactoring - there’s a better way. One of my go-to refactoring techniques is Parallel Change. It’s the same concept used in road construction: instead of blocking an entire street until the work is done, you build a detour to keep traffic flowing. Similarly, with Parallel Change, your code continues to function while changes are being implemented. If you’re new to this technique, start small. Practice with simple examples or katas to understand how it works. As you gain confidence, apply it to your day-to-day work - it’s a great way to develop the habit of keeping your code functional throughout the process. When dealing with modernization or legacy projects, this method proves its value even more. It eliminates the headache of fixing broken, non-compiling spaghetti code, allowing you to commit anytime and pause your work without worry. Mastering Parallel Change can make refactoring smoother, safer, and far less stressful. Give it a try - you’ll never want to go back to dealing with broken code.

Software engineers often underestimate how a single line of code can impact the company's profits. And it could be a trivial log line to print information for debugging. 😫 😫 Few years ago, my team was owning an AWS Lambda that worked very well and required minimal intervention. One day my Manager asked me why is the CloudWatch cost $15,000 but Lambda's cost was $1,200 only. 😱 😱 I decided to root cause this issue and finally figured out the main culprit was redundant log lines in the lambda. Eliminating the log lines bought down the costs by 10x. 🚀 🚀 What was the main issue for high CloudWatch costs ? 👉 CloudWatch charges $0.5/GB for ingestion and $0.03/GB for storage 👉 Our AWS Lambda was logging close to 5MB data per second. 👉 It was logging the request and a huge response payload (~100KB) 👉 As a result, the overall log ingestion cost was high. How did we debug the issue ? We used the CloudWatch log metrics to check the data usage. And identified the log group that was resulting in increased bill amount. CloudWatch console tool helped in debugging the root cause. How can we prevent such issue in the future ? ✅ Only log useful information i.e exceptions, critical errors, etc. Avoid logging everything. ✅ Use log levels such as Debug, Warn, Info, Error, etc. ✅  Add filtering to filter only the Error/Warn logs before ingesting into CloudWatch ✅  Review the code carefully and assess the impact of log line on the costs. Treat debug lines like a vulnerability. ✅  Continuously monitor the CloudWatch costs and set alarms to warn the team of any high costs. One of the key takeaways from this story is that engineers must know what impact each line of code will have on the overall business. And accordingly adopt best practices to prevent high costs. In case you have experienced a similar issue in the past, you can post in the below comments what best practices you are following. 👇 👇 #tech #aws #cloud #cloudcomputing

Use these essential techniques for crafting high-quality code in your data job. Here’s how you can ensure your code meets the highest standards: 1. 𝗙𝗼𝗹𝗹𝗼𝘄 𝗕𝗲𝘀𝘁 𝗣𝗿𝗮𝗰𝘁𝗶𝗰𝗲𝘀: Stick to the conventions and standards in your language of choice, whether it's Python, R, or SQL. This includes using meaningful variable names, intending your code into blocks, and organizing code in logical modules.     2. 𝗜𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁 𝗩𝗲𝗿𝘀𝗶𝗼𝗻 𝗖𝗼𝗻𝘁𝗿𝗼𝗹: Use tools like Git to manage changes and collaborate more effectively. Version control isn’t just for software developers but for every professional writing code.     3. 𝗖𝗼𝗱𝗲 𝗥𝗲𝘃𝗶𝗲𝘄𝘀: Engage with your team to review each other's code. This not only helps catch errors before they become problems but also promotes a culture of learning and knowledge sharing within your team.     4. 𝗪𝗿𝗶𝘁𝗲 𝗧𝗲𝘀𝘁𝘀: Although it might seem over the top for some data projects, writing tests for your code can catch bugs early and save hours of debugging later. It ensures that your code functions as expected and makes modifications safer and more reliable. You can include different tests like unit tests for functions and schema checks for your inputs.     5. 𝗥𝗲𝗳𝗮𝗰𝘁𝗼𝗿 𝗥𝗲𝗴𝘂𝗹𝗮𝗿𝗹𝘆: Revisit and revise your code regularly. Refactoring helps to improve your code efficiency and readability. As you learn and grow, you'll find ways to make your existing code better.     6. 𝗗𝗼𝗰𝘂𝗺𝗲𝗻𝘁 𝗘𝘃𝗲𝗿𝘆𝘁𝗵𝗶𝗻𝗴: Good documentation is not just about commenting within the code. Document your data sources, your algorithms, and your methodologies in an accessible way. This is crucial for when others (or you in the future) need to understand and possibly build on your work. A good place for this additional documentation is architectural decision records placed in your repository.     7. 𝗙𝗼𝗰𝘂𝘀 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝘀 𝗼𝗻 𝗢𝗻𝗲 𝗧𝗵𝗶𝗻𝗴: Make sure your function is doing exactly one thing. Having to use "and" in the function name or the comment describing the function then split it! It will make it much easier to understand and modify later on.     8. 𝗟𝗲𝘃𝗲𝗿𝗮𝗴𝗲 𝗟𝗶𝗻𝘁𝗲𝗿𝘀 𝗮𝗻𝗱 𝗙𝗼𝗿𝗺𝗮𝘁𝘁𝗲𝗿𝘀: Tools that automatically check your code for style issues, errors, or syntactic discrepancies can drastically improve your code quality. They enforce consistency and help you focus on the logic rather than the format. You level up your code from good to great by including these techniques in your workflow. High-quality code means fewer errors, easier maintenance, and more reliable results. What strategies do you use to maintain high code quality? ---------------- ♻️ Share if you find this post useful ➕ Follow for more daily insights on how to grow your career in the data field #dataanalytics #datascience #cleancode #python #codingstandards

The cheapest place to fix a mistake in a software project is on a piece of paper, not in six months of code. Writing a clear requirement is a great start. But the real skill, the thing that separates a good project from a great one, is actively trying to break the logic before you build it. Here at Full Metal, we call this pre-emptive debugging. We map out the "happy path," where the user does everything perfectly. But then we spend more time on the "unhappy paths." We ask a series of 'what if' questions. For a simple password reset feature, we'll ask: ❌ What if the user enters an email that isn't registered? ❌ What if they click the reset link after it has expired? ❌ What if they try to reuse an old password? Each of those 'what ifs' becomes a new requirement, closing a loophole that could have caused problems down the line. It's about finding flaws where they're free to fix. This also helps us avoid common pitfalls I've seen time and again. The biggest is the ambiguity trap: using fuzzy words like "fast" or "easy." My "fast" is not your "fast." Instead of "The system should be quick," we define it: "The system shall return a response within 500ms." One is a wish; the other is a testable fact. This meticulous approach might seem like a lot of work up front, but it saves a fortune in rework and frustration later on. We explore these common pitfalls and how to avoid them in our latest blog for SME leaders. Find the blog here: https://lnkd.in/eptHVTKA Have you ever had a project go a bit pear-shaped because of a single, unasked 'what if' question? #SoftwareEngineering #RiskManagement #DigitalTransformation

One of my favorite things about our Women in Project Management Network® #womeninpmnetwork#womeninpmnetwork community is how we focus on learning by doing... . . . Right now, we’re not just studying the...we’re in a simulation project putting it into practice using the knowledge we have learned and understanding some project management tool... Enter→ Azure DevOps and Jira Some of the same tools project teams use every day. Here’s how we’ve been approaching it: First, we walk through the SDLC phases to understand the flow from requirements to deployment (because we are simulating a Tech project) Then we take it into the tools, creating Epics, Tasks, mapping workflows, tracking tasks, and seeing how all the moving parts come together. The best part? That moment when one client sent me a message over the weekend sharing their "aha" moment... when the theory finally clicks because you’ve done it yourself. If you’ve been wanting to explore these tools, there are easy ways to get started on your own too: ↳ Jira: Sign up for a free trial on Atlassian’s website. They even have guided tutorials to walk you through creating your first project. ↳Azure DevOps: Microsoft offers a free account and free trial where you can explore Boards, Pipelines, and Repos, perfect for practice projects. Project management isn’t just about certifications or titles; it’s about having the confidence and hands-on skills to step into a project and contribute from day one. If you’re learning project management, which tool would you want to start with, Jira or Azure DevOps or Asana or Monday or MS Project or Smartsheet? Happy Monday!

Why Debugging Makes You a Better Engineer Writing features feels rewarding but debugging is where real engineering growth happens Because debugging forces you to: • Understand systems deeply • Read unfamiliar code • Think in edge cases • Stay patient under pressure In production, problems rarely come with clear answers Sometimes: • Logs are incomplete • Errors are misleading • The issue is happening somewhere completely unexpected And that’s where debugging changes your mindset Over time, debugging improves something more important than coding speed: engineering intuition You begin noticing patterns faster You ask better questions You think more systematically A lot of great engineers aren’t just good at building things they’re exceptionally good at figuring out why things broke in the first place

When I first learned Azure Data Factory (ADF), I focused on getting pipelines to run. What I didn’t realise? The real challenge is making sure they don’t break quietly at 2 AM. Here are 12 error handling techniques in ADF: 1. Try-Catch-Finally Pattern This classic structure lets you execute primary activities (Try), define actions upon failure (Catch), and specify cleanup tasks that run regardless of the outcome (Finally), ensuring a robust pipeline. 2. Activity Output Access the result of a preceding activity to make decisions. Use the expression @activity('ActivityName').Output to retrieve its JSON output, which is crucial for custom validation and conditional logic. 3. Activity Error Details When an activity fails, capture the specific error details. The expression @activity('ActivityName').Error provides the error code and message, which is essential for precise logging within a Catch block. 4. Retry Policy Automatically re-run a failed activity. You can configure the retry count and the interval between attempts, making your pipeline resilient to transient issues like temporary network failures or database locks. 5. Timeouts Set a maximum run duration for an activity. If it exceeds this time, ADF marks it as "TimedOut" and fails it, preventing a single long-running task from stalling the entire pipeline. 6. Validation Activity Proactively check for a condition before proceeding. This activity can verify if a file exists or if a query returns a specific value, failing early if the prerequisite is not met. 7. Fault Tolerance (Copy Activity) In a Copy Activity, configure settings to skip or log incompatible rows (e.g., data type mismatches) instead of failing the entire operation. This is essential for handling inconsistent source data. 8. Custom Error Logging Use a Script or Stored Procedure activity within a Catch block to write detailed error information to a log table. Capture helpful diagnostics like @pipeline().RunId for auditing purposes. 9. Alerting and Notifications Integrate your data factory with Azure Monitor to create alerts. You can configure action groups to automatically send email or Teams notifications when a pipeline fails, enabling prompt incident response. 10. Global Parameters Define environment-specific settings, like connection strings or logging levels, as global parameters. This allows you to manage configurations centrally, reducing errors when deploying across different environments (Dev, QA, Prod). 11. Modular Error Handling Use an "Execute Pipeline" activity to encapsulate logic. A child pipeline can have its own error handling and pass its final status (success or failure) back to the parent pipeline. 12. Data Consistency Verification After a copy activity, use a Lookup or Script activity to query source and sink row counts. This practice validates data integrity and helps you catch silent data loss failures.