Darren Kopp - On Software

Coding interviews are stupid (ish)

darren@darrenkopp.com (Darren Kopp) — Wed, 01 May 2024 00:00:00 +0000

I have, once again, failed an interview (probably). I don’t actually know yet but if I were the interviewer I probably wouldn’t recommend moving the candidate forward. Based on the hyperbolic title, you are probably guessing that I’m pretty salty about failing, but I’m not. I write one of these reflection blog posts each time I fail an interview I care about (fun fact: both were about the same company, enjoy the bonus round at the end). Now I’m writing one after experiencing a few years at a company that didn’t have any coding interviews (fun fact: I almost didn’t pass their interview process either).

Now, I’m just as guilty about being on the coding interview bandwagon as everyone else. I’ve given probably about a hundred of the same formulaic¹ interview that I just took. I’ve passed on most of those candidates and I would have passed on me. I’ve also spent hours fixing code that had issues directly stemming from not using the correct data structure/algorithm that I may not have had to if we had been better at screening the employees during the interview process.

So which one works better? From my small sample size and general mediocre statistics abilities, I’d say they are about the same. What I do know, however, is that for every 1-hour interview where I evaluated if someone knew their data structures, I could have just taught them. Maybe they did know them already and just forgot because they haven’t used them recently. Math was one of my strongest subjects in school but if you asked me right now to take a derivative of something I wouldn’t be able to.

For some reason we think that someone needs to be able to whip out the solution to any random problem that is space and time optimal within 40 minutes. Ok, that’s not necessarily fair (more hyperbole, I know). I’m well aware that it’s often said that what’s more important is how you solve the problem, not that you solve the problem, but in practice human biases will work against you if you don’t get close to it working.

If you’ve stuck around this far, you are probably curious about the question. Honestly, the question is not interesting at all and I thought it to be perfectly reasonable (and was probably the first of two). The reason I’m writing about all of this is because I woke up to the epiphany that the problem I failed was a problem I solved a year ago.

Q: Assume that you have an infinite stream of data that is coming in from multiple threads in an unordered fashion. Write the stream items to the console in order.

My actual problem, which took about a month, was that I needed to ingest all of the changes from one or more tables in SQL Server as fast as possible. What I chose to do was hook into the Change Data Capture functionality and simultaneously stream the data from CDC while scanning the table for snapshots of the data. Now, while each of those go in sequential order, we need to reconcile the two together depending on which we read first (ie if we get updates before snapshot, we need to store them until we get the full row data and then send them). On top of that I then built an in-memory transaction layer sitting on top of SQLite to optimize memory/disk usage to hit the performance benchmarks I wanted.

This literally checked off every box they were looking for (even the ones I missed) in the interview question. Thread safety: check. Lock primitives: check. Optimal Concurrency: check. Producer-consumer: check. Fault tolerant: check. Hashing: check. The code I wrote is ok at best, honestly. The greatest lie we have ever told ourselves is that we want greenfield projects because we won’t have to deal with legacy code, but legacy code is just greenfield code that is written under the duress of trying to solve the problem at the same time.

So now, just as when I’m in an interview, I’m at a loss for the answer to the question at hand: what should we do instead? I honestly don’t know but I do feel like we are stuck in a cargo-cult mentality where we are just doing things becaues that’s what the big companies do, and if it works for them it must be what we need to do. I understand the problems associated with hiring the wrong people as well, which may well be the actual reason we are rightfully stuck with such fearful/timid hiring practices.

For now, I’ll just keep practicing for interviews until I successfully trick someone into thinking that I know how to code and then secretely become one of the best employees they have ever had.

Bonus round! I guess enough time has gone by (11 years!) that I can say that both of my failed interview posts were from when I applied to StackOverflow. The first time I bombed out just as hard as this interview. The second time, however, I went through 2 - 3 coding interviews (which I believe that I passed), 1 interview with the product manager, and then finally an interview with David Fullerton and Joel Spolsky. I completely bombed the interview with David Fullerton and I’m pretty sure that’s what killed my application (but I can’t say one way or another).

That interview, however, is the only interview that actually bothers me to this day. The question, which I’ve heard is a Facebook favorite, was “convert a decimal number to base negative 2”. This question, which is possible/doable in 40 minutes and more about the problem solving process than anything, is the dumbest fucking question I have ever had and I don’t care what anybody else thinks. Every other question I’ve had I’ve never had problems with because, like I’ve shown in this post, most everything you’ll actually run into at some point. And yes converting numbers to other things (roman numerals, etc) is a valid thing, but fuck that question and just waiting for you to eventually arrive at the little trick to make it work.

Because of my schedule, the interviews were stacked back-to-back, where they normally would not be and I wouldn’t have actually ended up having the interview with Joel, which was so awkward because he was a little late and so I was just sitting there on a spinning google hangouts waiting while my head was still trying to process the question while knowing that I had completely bombed that interview and the next interview didn’t matter, not actually knowing if it would happen or not.

No hostility to any of those involved, but to this day I hate that question and it’s the only question I’ve received that I would actually categorize as “gimicky.” RIP StackOverflow Careers.

5 minutes on introductions, 40 minutes on the question, 10-15 minutes for questions. ↩

Faster Lambda Http Performance when connecting to AWS Application Load Balancer

darren@darrenkopp.com (Darren Kopp) — Thu, 15 Nov 2018 00:00:00 +0000

My current project at Homesnap involves breaking a monolith application into multiple microservices and part of that is moving a large amount of data to that new microservice by way of feeding the data through our new API.

While I’m able to get reasonable throughput by firehosing the API with data from my machine, it simply doesn’t have enough throughput to move the volume of data necessary in any reasonable amount of time, so I decided to queue the data using Amazon’s Simple Queue Service (SQS) with a Lambda function being triggered from data being written to the Queue. In theory, this allows me to scale out the data ingestion using Amazon’s capacity rather than me finding more and more machines to run my import utility on.

What I found, however, was that Lambda, with 80 concurrent executions sending 10 requests per batch, could barely outperform my single machine. My setup was pretty basic: a stateless, HTTP-based API behind an AWS Application Load Balancer (ALB); I would expect Lambda would achieve linear growth until the database’s resources were exhausted. So I began tinkering until I got the performance I was expecting.

1. Sticky Sessions

While I wouldn’t expect sticky sessions to be necessary (since my application is completely stateless), I wasn’t able to get reasonable performance from my machine or Lambda to my API through ALB without sticky sessions enabled. While this improved performance significantly on my machine, Lambda still suffered, so I kept digging.

2. Get node http to use sticky sessions

My Lambda function was just a simple node function that was using the node http functionality to send the requests to the API. However, unlike client-side javascript using the fetch API, the http library doesn’t automatically store / send cookies for the request. Thus, while sticky sessions were enabled, the Lambda function was never benefitting from those sticky sessions! Fixing this fact is fairly trivial, once you know what needs to be done.

When Lambda loads your node function, it executes your script once, but your exported handler function repeatedly until the process shuts down. We'll exploit this fact in the code snippet below.

const http = require('http');
// initialize our cookie to an empty array
let cookie = [];

function sendRequest(data) {
    return new Promise((resolve, reject) => {
        const request = http.request({
            host: 'example.com',
            path: '/api',
            method: 'POST',
            headers: {
                // Pass our cookies to the request
                "Cookie": cookie
            }
        }, (response) => {
            // when we receive a response, store the cookies returned from the server into
            // our cookie variable. note that a cookie is a semi-colon delimited string of properties
            // but the only part we want to send up to the server is the first part.
            let cookie = (response.headers["set-cookie"] || []).map(v => v.split(';')[0]);
        });

        request.end(data);
    });
}

exports.handler = async (event, context, callback) => {
    await sendRequest('{ "hello": "world" }');
};

The result? My function went from ~2,000 invocations per minute to ~9,500 invocations minute and the maximum duration of my function dropped from ~22 - 28 seconds to ~6 seconds. Additionally, I was able to reduce the number of docker images running in the cluster from 20 to 6 while sustaining the same throughput. All in all, everything is running faster with lower cost, which makes me happy.

Hopefully this can help you if you are in a similar situation.

The importance of computer science fundamentals

darren@darrenkopp.com (Darren Kopp) — Thu, 15 Mar 2018 22:09:00 +0000

Rob Conery has released a new book called The Imposter’s Handbook for those in the software industry who don’t have a strong background in computer science fundamentals. I haven’t read the book so I can’t comment on what it covers, but the concept has me reflecting on my own similar experience.

While I did attend some university, I only finished about half of my computer science degree before I dropped out. At the time, I didn’t think I had missed out on anything because I had already taught myself everything I had encountered in school up until that point; I was always a semester ahead of my course work.

I realize now that I was pretty close to the promised land of computer science because I had just hit Big-O notation though at the time I thought it was completely pointless. At the time, I was somewhat correct, but looking back now I realize how wrong I was.

This idea still resonates with many other self-taught developers though. The easiest place to see this is the criticism of technical interviews and their focus on rote data structure / algorithm questions. Often the counter-argument is “I don’t need to know this because I can just google it”, which is a fairly valid response.

So why, exactly, should you care about computer science if you are already a good developer? Well, you don’t, to an extent. You can be a succesful developer, at least for a while.

I was successful for years before I started diving deep into computer science on my own. I wasn’t some prodigy, I was just riding the effective application of computer science through my use of a DBMS. In fact, a large number of developers can, and are, successful because those fundamentals are baked into the frameworks that they are using.

At some point, though, you will start hitting scaling problems as you continue to be successful. Often times, the scale may not be as much as you would like. Sometimes, it will happen because you finally land that big client.

The classic example of this is you have a component that has some sort of double loop in it that works great with all of your existing clients who have a hundred items of something, and then you get that client who has a thousand, perhaps even ten thousand items, and everything suddenly starts falling over.

This is why many companies focus on ensuring that developers have a good grasp of data structures, algorithms, and their space / time complexity analysis during the interview process; they have hit the point where they can’t scale further without doing things right. Furthermore, they can’t afford to have people writing code that, when released, will immediately fall over under load.

So, if you’ve made it this far, I now make an appeal to you, reader. If you don’t have a strong background in computer science, start today. You don’t need it, but it can only make things better. Now is the greatest time to do such a thing because there’s never been better access to that information. Whether you read a book, take an online course, or read blogs, there’s a multitude of information out there for a reasonable price.

Conditional section rendering in ASP.NET MVC

darren@darrenkopp.com (Darren Kopp) — Tue, 01 Mar 2016 12:30:00 +0000

Recently I was working on reducing some redundancy in an ASP.NET MVC application and ran into an error when trying to conditionally render a section. ASP.NET was not happy and gave me the following error:

The following sections have been defined but have not yet been rendered for the layout page “~/Views/Shared/_Layout.cshtml”: “ProductionOnlyScripts”.

On this layout page we have the following code.

@if (ApplicationConfiguration.IsProduction)
{
    @RenderSection("ProductionOnlyScripts", false)
}

I never expected that if I defined a section that MVC would require me to render it. I can kind of understand why, but I disagree with this design decision. Lets run down the options we have with sections in general.

Calling @RenderSection(name) when the section has not been defined will throw an error. This is the right thing to do.
Calling @RenderSection(name, required: false) when the section has not been defined will not throw an error. This is the right thing to do.
Not calling @RenderSection(name) or @RenderSection(name, required: false) will throw an error if that section has been defined (as is what we have observed).

So, the work around to this is simple: we render the section to nothing.

@if (ApplicationConfiguration.IsProduction)
{
    @RenderSection("ProductionOnlyScripts", false)
}
else
{
    RenderSection("ProductionOnlyScripts", false)?.WriteTo(TextWriter.Null);
}

In the end, ASP.NET is tracking the section fragment and ensuring that we are writing out it’s contents. What it doesn’t know is that we are just writing that to /dev/null effectively so that the content will never make it’s way to the browser.

This can be extended as well into an extension method if you find yourself doing this often.

Always learn something from an interview

darren@darrenkopp.com (Darren Kopp) — Thu, 25 Feb 2016 12:02:00 +0000

The only real mistake is the one from which we learn nothing.
John Powell

Let’s make one thing clear: being a candidate in an interview is hard. Over your career you will likely participate in many interviews as a candidate, and I can assure you that you likely won’t pass every single one. You aren’t alone here, I myself am no stranger to failed interviews. You can, however, make sure that you are getting something out of every single interview: a new job, exposure to a new problem, a new way to solve a problem, or a better understanding of a problem.

As I have been interviewing candidates recently for a software engineer position at DevResults, I’ve been thinking about what I would do if I had been the candidate in these interviews and I’ve compiled my current thoughts into a few general tips. Some of these are technical interview specific, but the majority are generally applicable to all interviews in general.

Failing a problem is an opportunity to learn

One interview I participated in I was asked to return all permutations of a string. At the time, I had never had to do something like this, and needless to say I didn’t do well on it. After the interview I took the time to really understand the problem and strategies for generating permutations in general, mainly to ensure I wouldn’t ever fail that question again.

Fast-foward to last year, I found myself in a scenario where I did in fact need to generate all possible permutations for a set of data. It made me really appreciate that I had been exposed to that problem in the past from the interview and it was pretty trivial to apply what I had learned to solve the problem.

Most of the time, however, you’ll probably see something you’ve encountered before, or some variation of a known problem. In scenarios like these, there’s often multiple ways to approach a problem and if you didn’t quite get it, the interviewer may explain the solution to you or you can ask what the solution is. If you are able to get the solution to the problem, definitely take the time later to make sure you understand the solution and why it’s the right solution.

If you actually got the solution, but in a non-optimal way (perhaps your solution had poor runtime performance or poor memory usage), be sure to note the problems your algorithm had and research solutions that address those issues.

Ask for feedback

Don’t hesitate to ask for feedback or suggestions on how you did in an interview, but keep your question focussed on your performance. A lot of the time, you won’t get a response for various reasons like company policy prevents the interviewer from giving any feedback to you. Any feedback you are able to get, however, can be very important.

Every interview I conduct I block off an hour and a half which is broken into three parts:

The first 10 - 15 minutes I go over the job description and get to know a little more about the candidate
The next 45 minutes is dedicated to the candidate answering technical questions
The remaining 30 minutes is open question time for the candidate

The vast majority of the time candidates spend about 10 - 15 minutes asking the usual questions like what kind of source control we use, how we do deployments, etc. The other day, howerver, one candidate took full advantage of this time, which I always explicitly say is open Q&A. The candidate asked for feedback on their resume, how well they were communicating while they were coding, how well I could understand them (they were a non-native english speaker), and many other questions.

The insight was extremely valuable for them personally and likely not something any company policy would bar. Before and after each interview, think critically about what things you struggle with and try to get feedback from the interviewer on how they think you did in those regards. This will help you do better in all subsequent interviews.

The only way to get better at interviewing is to do interviews

This seems like common sense advice, but it really is true. If you are having a hard time with interviews, you just need to do more of them. Generally, people only apply to companies or jobs that they are really interested in and waiting to apply to other positions that they would likely be happy with but definitely not their first choice. This is fine, but it can be really demotivating when you don’t make the cut for positions you are really interested in.

When I’m job searching, I tend to take a balanced approach and apply to both types of positions, which allows me to practice interviewing and reduce risk of failing an interview with someplace I really want to work at just because my nerves got to me. It would be unethical to waste someones time by applying to positions you wouldn’t ever want though, so only apply to jobs you can see yourself being happy in.

The last time I went through the hiring process it actually became a little complicated in the end because I ended up facing a situation where I would have to choose between a job I thought I would really like and a position that when I had originally applied I thought I could be happy with but I knew I really wanted after I had met the people involved and learned more about the mission / vision.

Making posh-git more humane by improving the color scheme

darren@darrenkopp.com (Darren Kopp) — Sat, 12 Jul 2014 19:00:00 +0000

Out of the box, posh-git uses a red color that's a bit hard to read against most background colors. All hope is not lost, because it's very easy to fix.

The console on windows has some named colors built-in and posh-git is configured to use DarkRed. While we can’t change the named color that PoshGit uses, we can change the color that the console displays. To do this, simply right-click on the powershell icon, then select properties from the drop down menu.

It should be pretty clear which color in the row of colors is dark red, but if it isn’t then it is the fifth color from the left.

Once you select the dark red color, you can the selected color values to a more humane color. As can be seen in the screenshot, I’ve chosen a salmon color (R: 255, G: 154, B: 154) which goes well with my black color. Now, likely you’re still on Screen Background and the console background has changed to a salmon color, so just remember to re-select the previous background color before you hit Ok.

That’s it. Now hopefully you’ll have a more humane posh-git experience.

libsassnet nuget packages

darren@darrenkopp.com (Darren Kopp) — Mon, 26 May 2014 19:00:00 +0000

I just released two new nuget packages: libsassnet and libsassnet.Web.

A while ago I blogged about the state of Sass support in ASP.NET. Since then, a lot has changed: Sass hit version 3.3, libsass has become extremely popular, and Visual Studio 2013 now has support for Sass built in.

Unfortunately in the case of NSass, it hasn’t been updated in over a year. For that reason, I’ve been porting most of the functionality over to libsass-net, and the nuget packages and ASP.NET integration has been a glaring whole in that project for a while.

Pre-requisites

Since libsass is a C++ project, you’ll need to make sure you have the Visual C++ Redistributable installed. I’m not a C++ expert, so I’m not sure if the latest version works or if you need the 2012 version.

Integrating with System.Web.Optimization

Integrating with the System.Web.Optimization project is very simple with libsass. Below is an example configuration:

var sass = new SassBundle("~/content/css", basePath: "~/sass")
    .Include("~/sass/app.scss")
    .Include("~/sass/fonts.scss");

bundle.Add(sass);

Ignoring SignalR in NewRelic

darren@darrenkopp.com (Darren Kopp) — Wed, 07 May 2014 19:00:00 +0000

We just started using NewRelic at work and I’ve really been digging it. We also use SignalR which (by design) can use long-polling connections. Unfortunately, those tend connections to skew the metrics in NewRelic. Fortunately, NewRelic provides an API to ignore certain transactions, so I thought I’d be able to tell it to just ignore signalr.

At first, I tried this answer on StackOverflow, but it ended up ignoring all requests, not just the ones for SignalR. In the end I had to use an Owin module to get the job done.

Here is the code I ended up with

using System;
using System.Collections;
using System.Collections.Generic;
using System.Data;
using System.Diagnostics;
using AppFunc = Func<IDictionary<string,object>, Task>;

public class NewRelicIgnoreTransactionOwinModule
{
	private AppFunc _nextAppFunc;
	public NewRelicIgnoreTransactionOwinModule(AppFunc nextAppFunc)
	{
		_nextAppFunc = nextAppFunc;
	}

	public Task Invoke(IDictionary<string, object> environment)
	{
		object request = null;
		if (environment.TryGetValue("owin.RequestPath", out request)) {
			if (((string)request).IndexOf("signalr", StringComparison.OrdinalIgnoreCase) > -1) {
				NewRelic.Api.Agent.NewRelic.IgnoreTransaction();
			}
		}

		return _nextAppFunc(environment);
	}
}

To use this in your owin startup code call the following

app.Use(typeof(NewRelicIgnoreTransactionOwinModule));

Skill Atrophy

darren@darrenkopp.com (Darren Kopp) — Mon, 07 Apr 2014 12:00:00 +0000

Just like a muscle technical skills can and will diminish over time if you don't take the time to regularly practice them.

About 8 years ago I took a C++ course at Weber State University and everything was great. I had just finished my C course, so C++ was fairly straightforward and I didn’t have any problems with the language. I was even able to teach myself C# using what I learned from C and C++ and have been using C# ever since.

Since that time, I have not written any C/C++ code. Zilch. That is, until just recently when I needed to write some C++ code for libsass-net that would add support for generating sourcemap files. I didn’t even need to really write any code, just hook into the existing code. Needless to say, it wasn’t that simple.

Problem after problem

Before I was even able to get started coding, I had already hit a problem after pulling the latest changes from libsass. The compiler kept complaining about a method named UNICODE, but I couldn’t figure out what the problem was. The only hint I had was that Visual Studio was highlighting this method differently than all the other methods.

After a while I finally figured out that a configuration setting in my project file was telling Visual C++ that I wanted to use Unicode strings, so there was a #define UNICODE being emitted by the compiler. I believe I may have found the solution quicker if I had been more familiar with the toolset.

Throughout the process of trying to figure out why things weren’t working I found myself having a hard time understanding what most of the code was actually doing. C++ is a very powerful and very terse language; I found myself struggling to keep track of all the symbols that were in front of me. I was lost in a world of template methods, void * pointers, and lots of other features I vaguely remember.

Keeping up to date

For me, I have decided that I should spend a little time to maintain at least a reading level of the various languages that I have learned over the years. I know that I won’t be able to easily maintain a fluency level in all the languages I have learned because I won’t be writing in them daily, but I can at least retain some of my skill set by reading other people’s code.

In fact, I think by maintaining the libsass-net project, I am in an ideal situation: I am required to write a minimal amount of C++ and need to have at least a minimal reading comprehension of the language. Hopefully I’ll be able to keep my skills somewhat up-to-date without having to go overboard and force myself to make my own project to practice.

Regardless of how you approach the problem, be aware of the skills that you are letting atrophy and take the time to practice them if those skills are important to you. If you do not, you may find yourself in a similar situation to my own, and it’s not the most pleasant place to be.

Slides from my Utah Code Camp session

darren@darrenkopp.com (Darren Kopp) — Sat, 15 Mar 2014 07:00:00 +0000

I had the pleasure of presenting a session on SASS for Utah Code Camp. It was a great event and I’m glad that I was able to participate! Below is the slide deck for my session “Supercharge CSS with SASS”