agile, Career, Product Management, Project

Changing the process from within

Would you agree with the statement below?

If you disagree with how something is done, you should give the process a try and drive changes from within.

Give yourself some time to think before reading the example below.

I was given an opportunity to drive some works that are normally done by Lead Developers/Team Leaders/Tech Leaders etc. I initially rejected it because I didn’t agree with that specific process. It is a discovery process where the engineer lays out options, considers the pros and cons, considers the use cases, proposes a solution, and gets a written agreement from the interested parties. Let’s dissect this process with a real example.

Example: Cross-account Kafka applications in AWS

  • Goal Summary: Source data from an application database in one AWS account to a MSK Kafka Cluster in another account.
  • Options: There are a few layers of decisions to be made. The publishing technologies could be custom publisher, Kafka Connect, generic in-house publisher. Then there is the cross account networking challenges. A proxy, direct connection, clone the DB etc. And then there are language choice, data flow design etc.
  • Decision: Kafka Connect with Network Proxy
  • Decision Maker/Interested Parties/Influencers: Product Manager from team A/Team B, C and X/ Team O, P and Q

This is an extremely over simplified version of the process. It took the team a month to finalise the proposal from trying out different options, to coordinating with different teams and many many discussions. But here is my interpretation of this process: we have a big challenge in front of us and we want to build some certainty before we step too deep into one direction. We also want to get an agreement between the teams that are involved, either in building them, might be using them, or potential maintenance team.

Why I disagree with this process

Retrospectively, I think I am too much of a purist when it comes to software development process. I have this ideal state of how Agile teams should be operating. Trust me, we are an Agile organisation but it is difficult to convey what that means nowadays.

First of all, the process is a waterfall in disguise. If it sounds familiar to you, the documentation that it produces, the contract model, the desire to be certain; they are very similar to the classic software specifications. For the record, there is nothing wrong about waterfall because it is still a great model for certain types of softwares. However, this process is so much leaner than a standard waterfall process. The document is in Git Issue and it was created with the understanding that things will change down the road.

Secondly, the reality is often not as good as the idea set out to be. The structure of the process is such that a single person or two mull the problem for a long period of time before presenting to the stakeholders. Although others are encouraged to challenge and be critical about the options and decision, there are a couple of problems. One being that not everyone will be on the same page or same level of technical understanding of the problem to challenge properly. Often people will just shrug it out and agree with whatever is decided. And then there is the biases of the individuals that will influence the decision.

And then there are two common outcomes at the end of the implementation of the decision. Teams tend to stick to the plan even if the road is bumpy and might not fit perfectly. This is sometimes accompanied with comments like “but we agreed to”, “we have invested in the decision”, and perhaps “the other teams are expecting this to be X”. The second outcome is that it changes and changes and the decision is not relevant anymore but the documentation and comms never really get updated. Although it is a preferred outcome nevertheless, why spend the time in the process anyway?

An iterative approach

Not sure if it s specific to Scrum or maybe it has a specific name to it, but let’s call it Agile for the simplicity sake. We know that the idea of Agile is to build small, try it out, showcase, learn and adapt to changes. Too many buzzwords, let’s use the same example above and see how we can iterate through it.

First we need to break the problem down into smaller more manageable problems. We can do this during refinement where the whole team will be there. Product Managers will focus on representing the customers and work with team to see what values can be delivered in the next week or two. There is no right answer for sure but the first goal can be to build a cross-account connection to first allow us and others to build applications outside of the cluster account. Let’s set that as a goal. The team discuss the options and agree to try on one.

A week of the team getting their hands dirty in the implementations, does it work? can we present something to the customers? If it works, that is great and move on to the next challenge. If it doesn’t, we learn and inform the next decision. A few iterations later, we have a living system that is not necessarily done or perfect but it is delivering value also we know that we are satisfying the requirements because we keep in touch with our customers.

This process does not guarantee the perfect solution at the end either. But what I value most here is the learning. Although we might have to do more actual work than the previous process, we do it as a team and we also learn how to adapt. In the Evolutionary Architecture principle, it is about supporting change and not about making the perfect decision. By involving in the change process, we learn how to change better.

We also build a team with thinkers and decision makers, and not just builders. I believe software development is a creative process and it often is not enough to just build things, but also to create things.

What do you think now

So we were here because my manager thinks that I should give the process a try to be able to understand the value from the inside and also drive changes if needed to. Do you agree?

For me, I am in a protest. I think that the process is against my ideals and I would not just do it so I can progress in the career ladder. This is definitely not looking good for my career progression but hey! it is about the journey, not just the destinations. I am enjoying my role and I am still learning and growing where I am as a senior developer. Although we know that there are pressure in climbing the ladder (perhaps a story for another time), I am not in a hurry to get anywhere just yet.

Standard
Career, Product Management

Why a Software Engineer might not get that Product Management role

Interviewing is a skill. Often you might think that you checked all the boxes from the job description but still ended up failing miserably in the interviews. There could be a few reasons that you failed like you suck at answering interview questions, you are not actually qualified, or you might have prepared it incorrectly.

Suck at answering product management interview questions

Interviewing requires practice and preparation. Maybe you can still get away with it if the last time you interviewed was 4 years ago and you are applying for the similar role. However, applying for a new role is pretty much starting from scratch without a lot of past experience to carry you. You won’t have the day to day examples to present and you don’t have experience interviewing for that specific role.

Even if you were given the list of questions before the interviews to prepare, you might not actually answer it well. Let’s take “How do you manage competing priorities?”. The answer could be “to understand the value and cost of each piece of work and visualise it in a roadmap. You can use tools like RICE, Kano model, Value vs Efforts to help with estimating your projects”. All of these are “correct” answers, but what interviewers really want to know if you have done any of those yourselves. Without a real experience in Product Management, it’s almost impossible.

One can try to draw examples from engineer’s day to day. For example, you can talk about Sprint Planning and how you estimate works, visualise it and make decision based on the key criteria. For example, if something requires a few hours but will save the developers hours of manual task every week, that’s a quick win and should be done as soon as possible. While a big task that bring smaller values can wait.

You might not be qualified for the role

You don’t always know what is required by the job even if it has a very detailed job description. Every interviewers has their expectation of the role and it doesn’t have to be the same as the job description. And I think interview is not only a great way for a company to get the right person for the team but also for the candidate to clarify the exact role description.

Nevertheless, you might walk into the interview and realised that you are not qualified for the role. Just like a software engineering role where every team needs different skillset. For example, one might be looking for a higher tier senior engineer who can upskill very quickly and start coaching others. You might be a senior developer with very strong technical skill but not really invested in coaching skill and that’s just not a good fit. You are still a great developer for the right team, but just not for this team.

Preparing it the right way

As a software engineer, interviews are mostly technical. You know how to code, how to draw system diagrams, how AWS works. How well you perform in your interview depends on your experience and being prepared with good examples. You might be asked to explain a system, for example. And you would refamiliarize yourself with one of your systems and even draw a diagram from memory one or twice. Product management interviews are a lot more functional. How do you manage stakeholders, how do you sell the product, how do you manage senior managers, what is your vision.

On hindsight and after listening to one of the interview tips from Modern Product Management podcast, I learned that I should have role-played the role before the interview. The gist was to know as much as you can about the team and product, plan out the 30/60/90 days, map out the stakeholders, make some showcase materials and just working on the product. This will help put you in that role and be able to answer the questions more from a first-hand experience rather than basing on nothing. It is also a big bonus to show that you have done a lot of work to know their company.

The we vs the I

We often look for great team players when it comes to software engineers. And we are trained to almost exclusively use “we” instead of “I”. It becomes very important when interviewing for a product management role to know when to use “I” and when to use “we”.

For majority of the questions, it’s about what you do. So I’ll say that prepare to use “I”s more to show that what exactly that you did to achieve those goals. Often it’s just the product manager that’s talking to different groups of customers and juggle between their request, the team might not know about them until it’s at the top of the product backlog. So is managing the senior stakeholders, your team don’t usually go with you to those meetings. So take the pride and own your stories.

And there will be times to use “we” and it’s important to know when. For example, it is important that you give your team the credit when it comes to building a successful product. You might also need to tell a story of how you sell the dreams to your team, how you collaborated with other roles in your company like the engineers, designers, delivery managers, team leaders etc. Although product management can be a lonely role, you are still a part of a team.

Standard
Career, Product Management

A developer applied for a product manager role

I am a software developer. Throughout my career I have been that person who is technically capable but truly shine in the leadership skills. I champion Agile processes in every team I have been to, manages stakeholders well through proper expectation managements, and a great mentor. So seeing how I excel more on my soft skills than my technical skills, I thought product management is a perfect role for me.

I don’t think that was a wrong assumption. I still believe that I will be an excellent product manager. Nevertheless, I failed miserably on my second attempt at the role. So here are some wisdom to share on the cost of my hindsight.

Expectation of the role

Product management in general is about being the representation of the customers and the market. But every role is different even if they have the same title. The role I applied for had some expectations that I wasn’t expecting and might even disagree. But nevertheless as the discipline evolves, these responsibilities get more standardised among companies.

The biggest unexpected element for me was that a product manager is expected to be a visionary. I mean, it sounded like an obvious responsibility but I wasn’t expecting it to be a character trait rather than a process. Part of the interview conversation was to see if I had a vision for the team. In a deeper sense, it was a test to see how much I understood the domain, the market for this domain and if I can propose a plan. This presented a challenge for me because I didn’t know that being a visionary was a big part of product management. I came in with the expectation that a vision is made not born. I thought that it is created through a process of understanding the users, understanding the needs and the wants, defining the problems and with the creativity of the team to produce a goal. But coming in with a solid vision can be a big winner during an interview.

Next is about saying no. Product managers often jokes about saying no more than saying yes. My approach was about data driven and being diplomatic with stakeholders. But I felt that there were an expectation of a more assertive approach. As a developer I like to discuss things. Every disagreement is a conversation. And most of the leadership training talks about influencing through understanding what the other parties want and build a solution. But in a role like product manager where you can be constantly asked for things and everyone want it now, the answer that senior managers want to hear more is probably “no”. Just “no”.

Selling the product is another key aspect that I didn’t do well. The question I was asked was how to get buy-ins from investors or senior managers who haven’t heard of certain technology (Kafka) or don’t understand why they want to invest in your team. Expectation here is to be able to show how you can manage senior stakeholders. As I haven’t done much of those in my career it was difficult to draw from experience. But one way to approach is to use tools like Lean Canvas. The key here seems to be about being high level and show the value of what you are trying to achieve. You also need to be able to zoom out using the language that different group of people can understand. For example, senior managers might not understand or need to understand the intricate detail of your solution, but they are more interested in why you want to build it, how much it’s gonna cost, and how much money it’s gonna make. And doing all of these with enough data to back them up and to convince the stakeholders.

So what is next?

Failing an interview puts you in a weird place. I was really grateful that the hiring manager was someone I knew and worked with before, and they gave a very honest feedback after the interview. However, it was also very brutal. I definitely felt horrible afterward and ashamed for a long while. They were great feedback nonetheless and it opened my perspective more into this role. So moving forward is important but I’m still on the fence if I want to pursue this career path.

And even when you are at a great place where you feel supported, have a good feeling about where you are heading in your career, and getting a lot of great feedback on your current role; failing really put your failures on loudspeaker. It was really hard to not feel like an imposter, that everything is not as good as it was, or maybe you are not good enough. This is all a normal emotion and it’s important to acknowledge them. It is also very important to be kind to yourself. Imagine if you have a friend who have just failed an interview and feel like the most useless person in the world. What would you say to that friend? Can you treat yourself like treating someone you love who is going through a tough time?

Standard
Uncategorized

Agile process as a product

I am a huge believer of Design Thinking and it is gaining some traction outside of the UI world. But it’s still pretty unknown by the programmer bunch. Often when thinking about UX ( User Experience ), people think about Graphical User Interface (GUI). It’s almost taken as synonymous. But an UX designer would certainly disagree with that.

An UX professional can be from an artist, a programmer or an analyst background. But one thing they need essentially is the understanding of users. Because the users are most often human, an UX professional needs to understand human behaviours. It’s probably a stretch to call them an expert in Behavioural Science, but I think they’d think so.

This got me thinking about the time I was studying User Experience Design and Human Computer Interaction (HCI). More specifically the Shneiderman’s “Eight Golden Rules of Interface Design”. The golden rules are designed around the limitation of human’s limitation to process information. And that makes me wonder if Agile frameworks are built on those rules. Let’s use Kanban as an example and see we can map them to the rules.

The golden rules in practice

Strive for consistency

Consistent sequences of actions should be required in similar situations; identical terminology should be used in prompts, menus, and help screens; and consistent commands should be employed throughout.

Kanban has the “pull” system in conjunction with the “Limit WIP” principle. Both elements has the consistency built in it. Pull system dictates that you can only pull from left column to right column. this is consistent throughout the board. While the limit is defined to WIP, it’s applied to all columns including todo column. But I guess Icebox and Done columns ( when used ) are the exceptions.

Enable frequent users to use shortcuts

As the frequency of use increases, so do the user’s desires to reduce the number of interactions and to increase the pace of interaction. Abbreviations, function keys, hidden commands, and macro facilities are very helpful to an expert user.

I guess this can be a more philosophical than actual application of tools. What I can think of is the use of continuous improvements to improve efficiency of process. The process should become easier to follow and more efficient as the team members build more trust and better relationship. If it doesn’t, a tool such as retrospective can be employed to facilitate improvements.

Offer informative feedback

For every operator action, there should be some system feedback. For frequent and minor actions, the response can be modest, while for infrequent and major actions, the response should be more substantial.

Visualisation is a big part of Kanban. The board is meant to give feedback on progress. How busy are we?. What happen if we pull this here?. We have too many tickets in QA column. When we do something about it we can see the board changes. When we are closer to the end of an iteration we can see the todo column drying up.

Design dialog to yield closure

Sequences of actions should be organized into groups with a beginning, middle, and end. The informative feedback at the completion of a group of actions gives the operators the satisfaction of accomplishment, a sense of relief, the signal to drop contingency plans and options from their minds, and an indication that the way is clear to prepare for the next group of actions.

Tied together with visualisation, you can start to see the importance of closure. A closure is achieved when a card reaches the end of the board. When the todo column finishes. When a card is pulled out of a column. Scrum do Sprints and pretty much reset every iteration to yield closure as well, indicating an end of something.

Offer simple error handling

As much as possible, design the system so the user cannot make a serious error. If an error is made, the system should be able to detect the error and offer simple, comprehensible mechanisms for handling the error.

Before error handling is the error detection. JIRA has this cool feature that will highlight columns that broke the limit. The growing number of tickets left in the todo column at the end of an iteration will indicate problems to the flow. When a ticket stays in a column for too long.

Process design to handle these problems should be simple. When a limit is broken, you shouldn’t pull more items into it. When there’s many leftover, take less tickets during the next sprint planning.

Permit easy reversal of actions

This feature relieves anxiety, since the user knows that errors can be undone; it thus encourages exploration of unfamiliar options. The units of reversibility may be a single action, a data entry, or a complete group of actions.

I can’t think of any Kanban specific tools for this, but Continuous Integration is pretty much what this is trying to cover. Having a good test coverage and an easy and continuous build and test setup such as Bamboo or Buildkite will help to improve confidence. And the ability to rollback is a pretty much the Undo button in every good UI design.

i like this rule very much because it doesn’t trust human being to do the right thing all the time. It acknowledge that shit happens and it shouldn’t be human’s job to constantly looking over their shoulder. It should encourage creativity instead.

Support internal locus of control

Experienced operators strongly desire the sense that they are in charge of the system and that the system responds to their actions. Design the system to make users the initiators of actions rather than the responders.

When I read this rule I think about micromanagements. The pull system enforces autonomy of picking up a task rather than being assigned to a task. You can still be told to pull a ticket, which is bad. But when someone is ready for a next task, without the intervention of the managers, they should be able to confidently know what needs to be done next.

To achieve that, a few things needs to be done first. Number one is to set priority as a team, and with that the prerequisite of everyone’s understanding of what the stories are through sprint planning. Next is the explicit rules such as “left to right, top to bottom”, and “Pull” system to act as the GPS, And there also need to be enough trust in the team so anyone is comfortable at making that decision for themselves.

Reduce short-term memory load.

The limitation of human information processing in short-term memory requires that displays be kept simple, multiple page displays be consolidated, window-motion frequency be reduced, and sufficient training time be allotted for codes, mnemonics, and sequences of actions.

The core of this rule in the context of process design is the effort to reduce multitasking and context switching. Not sure if it’s exactly Kanban’s principle but the general rule that my team ran on is a person only can be assigned to one ticket at a time. Kanban helps to visualise that by putting faces on cards. In a physical board, we can limit the number of face stickers in the reserve. But this constraint can be easily spotted during standup.

In the GUI sense, the board is encouraged to be simple. Reduce the number of columns. Smaller team size.

Process Improvement Plan

Process design is important. But every team is composed of different unique individuals. That makes it impossible to create a silver bullet framework. It can be difficult to start from scratch. Scrum is a good place to start if your team is new to Agile. Kanban’s first principle is to start with what you do. The next and most important step is continuous improvements. This IMHO is the core of the Agile movement. It is a never ending process of improving the process. So instead of designing a silver bullet process, we should design a process improvement plan.

But we often see the opposite in the real world. In Agile methodology, we often look at processes and tools as something to avoid. It’s understandable because the Agile Manifesto defines that we should value Individuals and interactions¬†over processes and tools”. This value is often used as an argument against spending time talking about tools and processes. On one hand, this is a good interpretation of that value. It’s important to acknowledge it’s more important to spend time on improving interaction and individuals rather than following a set of rules. On the other hand, neglecting process could sometimes harm the interaction and individuals aspect of a team. In some of the well established frameworks like Scrum and Kanban, rituals and guidelines are often designed around people. They could help drive the interaction and individuals. But it is understandable that it can become a process pain.

Let’s take estimation for example. Scrum uses Poker Card estimation. Planning staffs often think that it’s a time consuming process with no much benefit. One of the Agile misconception is that it needs to be fast. And when it comes to estimation, people only think about that final number they need to tell their customer. When can we deliver this? Give me a ballpark! So some teams take the shortcut and get them from the team leader.

Let’s explore a little bit of a Scrum estimation ritual. Each member get a set of cards with fibonacci numbers on them. They represent “complexity” of a task. Someone, usually the Product Owner, would describe the task/project/feature/user story. Team members would then ask questions and discuss about the card. And everyone would secretly pick a number from the poker cards and reveal them at the same time. If the votes varies greatly, the outliers would need justify their votes. Revoting will happen until everyone’s happy with the estimate.

There are so much more details that goes into a simple process of estimation. There’s one about picking the baseline size, breaking down big cards, complexity vs time, equal opportunity. But most importantly for me as a developer, it forces every member to contribute and understand the story we’re playing to be able to vote confidently. It forces people to interact so they can have a confident vote.

Understanding reasons and benefit of each rituals is crucial in process improvements. Every conversation should start why do we do it and why doesn’t it work. With the help of the golden rules we can drive the discussion toward a process that is easy to use and one that people want to use.

References

Standard
agile, culture

Agile – From a developer’s eyes

What does it mean to be Agile? There are 3 common answers that I’ve summarised:

  1. It’s a corporate bullshit.
  2. Everyone interprets it differently, whatever works for you.
  3. You’re doing it wrong.

Somewhere between those lines, there’s a subset of people who truly believe that Agile is helping them so much in their daily life. I’m one of them.

I’m afraid I might have over-simplified the common sentiment of Agile in the tech community. However, what I’ve loved so much about this community are the people that have shown me how great Agile can be in the best way possible by bringing me in their journey, seeing the values for myself, and structuring frameworks and environments where I get to explore safely and help build a better culture.

And that’s one of the value in the Agile Manifesto, to value “Individuals and interactions over processes and tools”. We value individuals as human beings and trustworthy. We also value the interactions between individuals more than creating sets of rules or using specific set of tools.

There are many Agile writers out there that would be more comprehensive and most certainly helpful. But I’ve not found many that doesn’t focus on “delivery” or “customers”. As a developer who’s worked in various teams claiming to do Agile for over 5 years, and if you are also a developer, you would agree that most of these articles are irrelevant to us.

I’m not arguing that the customers are not the highest priority, they are. What I’m trying to say is that we’ve done so well in satisfying the customers we sometimes forget about the people behind it.

People are my passion. Human mind intrigues me and it’s my daily amusement to learn about myself, others and the interactions between. In this series, I’d like to explore on how Agile can help me as a developer and make my job less miserable. They’ll be focusing around people and not just developers but from a developer’s point of view. Stay tuned for the next one.

Standard
culture

On workspace design and work culture

Despite being labeled as “hard to change”, culture is actually a very fragile thing. You’d often hear people say that it’s hard to build a culture. But also true that it’s difficult to maintain a good culture. So which is true?. I also think that workspace design has a subtle but important effects on shaping culture at work. let’s explore some of the examples of the modern workspace design and how they can negatively impact the culture.

Hot desks

Hot desking is when the employees are not designated with a fixed seat.

Boosting competitiveness and selfishness

Where one sits depends on how early they get there every day. Most of the time, their seat is reserved for them, their team mates know where their preferences are and I’m surprised to find that most of the time I actually find my seat to be empty.

However, when challenged with expansion or on team movements, the seating arrangement will change and slowly become more settled again.

It sounds like a good approach, but what culture are we really building here? The obvious good one is flexibility. Also the flock mentality that dictate that the people who works together will move in a similar fashion.

On the downside, it’s also building competitiveness. Not every desk is the same. Some has better monitors, some has better ergonomic. Often when complaining about not getting a good seat, the answer is “well, you should have came earlier”. But is that the culture that we wanna have in a productive environment?. There will always be someone that’s last to come in and will end up with the less idea desk. That person, even it’s not you, is someone who is a valuable member of the company. Having the setup that build a culture that put others in a less ideal position for one’s benefit is simply toxic.

Stripping you off you

Yeah sure, it sounds like a good thing, having everyone in the same behaviour. No conflict, no war.

No, it’s not that simple. We as an individuals are innately different and there’s that natural instinct to be able to tell one from another. Having a hot seat system means there’s no way to customise one’s desk. Every desk looks the same. There’s no way to fashion a photo of the loved ones or having fancy productivity boosters like custom keyboards or funny memes without the price of packing and unpacking them everyday.

Culturally, it’s creating a bunch of robots without creativity. Not allowed to customise their desk leads to the feeling that we are not meant to change anything, that self expression is not encouraged.

Employee engagements

Familiarity can be a powerful tool. People are drawn toward nostalgia for example, that forgotten familiar thing of the past. Hot desking doesn’t allow that to happen. Before one has settled in they have to move away. Not just the desk they are siting on, but the surrounding environment like the people next to them, the location from the toilet, the morning walk from the front door.

When a simple daily task is not familiar everyday, it can feel unsafe, like sleeping in a hotel room or being at a friend’s house for the first time. It’s difficult to feel at home when one is not familiar with it or to be able to build a routine.

Culturally, it’s building detachment. It doesn’t allow an employee to be fully part of the team. And when we don’t feel at home, we don’t feel safe.

Open Plan

Open plan office is when an office has no barriers between employees.

Diversity buster

The most common complained disadvantage of open plan offices are the noise. It’s common that conversation will pop up easily anywhere. It could be a good thing to encourage communication.

However, for some individuals, this form of communication is less favourable for them. Communication shouldn’t be forced but should be natural. More often, the lack of the safe space is the hindrance to communicate. Open plan offices doesn’t solve that problem.

Culturally, it’s discouraging diversity on communication preference. Everyone is expected to be loud and clear, or to be interacting frequently.

Ignoring trust

Supervision is one of the benefit of having an open plan workspace. Having each other seeing what one is doing all the time is meant to be more productive because it reduces distraction and personal stuff.

But it also destroys trust, or neglect the need of one. You don’t need trust if you can see them all the time.

Team work when it’s not needed

It’s nice that most workplaces have great and kind people who are happy to help when needed. Sometimes collaboration will only be effective if everyone is in the same space.

However, even in an open plan, one can pass their day without talking to anyway but still be productive. Friendship is often forged from collaborations and meetings which often happen inside a closed space or outside of the office.

It’s also annoying to disrupt someone when they have their headphones on or when they’re concentrating on something. And sometimes people are actually annoyed by one’s constant nagging for help and most people don’t hide it well.

Culturally, it’s building insensitive. It’s encouraged that one shouldn’t be afraid to walk to another person and disturb them. And not just the person we are interacting but other people around us that could be distracted by things that we do.

I guess what I’m trying to say here is that we should be more aware of some negative impacts everything can bring in a culture. Most of these impacts are not a showstopper. But it’s important to acknowledge them and try to mitigate them when possible.

Standard
Uncategorized

Debugging PHP the hard way

Having to maintain a massive Object Oriented web application written in PHP, it’s amazing how many times I’ll have to use the same functions over and over again in multiple different places. I’m listing them down here before I forget.

error_log($message)

The most used one, and the only way I can print anything right now in the application I’m working on without screwing up everything. It prints out the message into the application error log, which I then tail -f and keep it running in one monitor. Way to go multiple monitors.

get_class($object)

How do I live without this. Even with the power of IDE ( I’m using NetBeans ), it’s frustrating sometimes to find out what the class of the object actually is. Especially in “well designed” object oriented application full of decorator and factories, it’s easy to get lost. get_class() produce the exact class of the object. Can’t live without this function.

var_export($object, $return=false)

var_dump() used to be my ultimate savior, but with the limitation on the printing into the web without breaking anything, I’ll have to print everything to the log file. var_export() does exactly that when the “return” flag is set to true. Although most of the times it ended up with recursion error, it’s still pretty useful.

Exception()->getTraceAsString()

Got to find the way to trace where something started. Pretty straight forward when combined with var_export() and error_log() as mentioned in one of the brilliant StackOverflow answer.

$e = new Exception;

error_log(var_export($e->getTraceAsString(), true));

What are your favorite debugging snippets in PHP?

Standard
Uncategorized

Sharing objects between Modules in AngularJs

It turns out that sharing objects between angular modules is as simple as injecting a module into another module. Say myApp has some controllers and providers or other objects and myApp2 want to use some of those controllers. When creating myApp2, inject myApp into it and myApp2 now has access to all the objects in myApp to the extend that the HTML can use the controller from myApp without any extra codes.

// Common library of some sort
app = angular.module("myApp", []);

app.value('someVal', 'This is a val');

app.controller("controller1", ['$scope','someVal', function($scope, someVal){
$scope.someVal = someVal;
}]);

// My actual module
app2 = angular.module("myApp2", ["myApp"]);

app2.controller("searchController", ['$scope', function($scope){
// ... some controller's codes
}]);
<body ng-app='myApp2'>
<div ng-controller='controller1'>
{{someVal}}
</div>
</body>

Note that the HTML is using ‘controller1’ which is from ‘myApp’. I’ve built an app in AngularJs and going to create another one using the similar format. It also means that most of the codes are going to be shared. I’m glad that I don’t have to do much refactoring.

Standard
Web Tools

Methods of getting direct feedback from Servers

With the web technology advancing so rapidly and information is getting bigger and flowing faster than ever, many web applications nowadays can’t live without constantly checking for new data from the servers.

The most basic form of a websites deals with HTTP request from the webpage to the server. The user send a request in an url form and the server response with the content that was requested. End of story.

Say that you are viewing a page that tell you how many times a jumping sheep has been jumping and it is jumping in average of once every 1 – 10 minutes, you might want to refresh your page every few minutes if it has been jumping or not. But with today’s technology, there should be a way that we get notified whenever the sheep jumps.

The main challenges of getting a direct feedback is that web application usually support a huge amount of users and they could be located anywhere in the world. It is simpler to think that the server just give the information whenever it’s requested. Instead of being busy trying to send updates to every connected clients (if they are connected). However, let’s explore the possible implementations.

Auto refresh

If you’re building a website in the 1990s, this is probably a very viable options. It’s simple to implement and logically sounds. The requirement is simple, you need to know when the sheep in your server jump, but the server is not capable of getting to you and every one of you. So we set up a simple Javascript to auto refresh the page every few seconds.

The drawback is, page refresh is rarely favourable in the fast moving content packed web applications. By refreshing the page, you will need to refresh other resources needed where we are only interested in a simple integer that tell us if the Sheep has jumped. So, let’s move on.

Ajax Long Pooling

Ajax open up many great possibilities that until this day has become one of the core protocol in building responsive web applications. With Ajax, we can send a request without refreshing the page. This also enable us to implement a much more efficient auto refresh that doesn’t request the full page request. Instead, we send a smaller HTTP request in the background and update the page using Javascript.

In addition to that, we can set the connection to stay alive until we get a response (or time out). Anyhow, this will still involve the web browser to constantly sending new requests every now and then to get updated. One drawback is that the server might be idle for few hours without any new worthy updates, so the resources used to send those requests might just end up in waste.

Web Sockets / Comet / Probably other terms

lastly we have web sockets. With the web servers and browsers needing more and more frequents interaction, there is finally a way that the server can be event driven instead of request driven only.

Web Sockets allow a more interaction connection where the server can be allowed to send response based on events that happens in the server itself. In this way, the web browser (client) doesn’t have to constantly send request for updates, but simply be event driven as well that only reacts when needed.

This will also reduce the overhead of constant new requests that happen between the client and server.

However, web sockets may cause some compatibility issues depending on the server technologies used and also the browser. But we’ve seen an increasing support for this technologies that allow developers to create more and more responsive applications.

One popular technology that works well with Web Sockets is Node.JS. But there are ways to work around Django through Redis or other library that can provide us with this capability.

Standard
PostgreSQL, Python

Reduce SQL Injection Risk in Python and psycopg2

It will be surprising that a slight different in your line of code can have a great impact in preventing SQL injection. Try to spot the difference between these 2 lines below.

# code 1 (with python string format() function)
db.execute("select * from some_table where filter = '{}'".format("going to Sam's party"))
# code 2 (with psycopg2 sql prepare syntax)
db.execute("select * from some_table where filter = %s", ("going to Sam's party"))

It probably looks obviously different, but for a second it looks like it shouldn’t give much different result. But sadly it does.

The first code use the python standard string formatting feature where given a string containing curly brackets as a placeholders like “This is a¬†{}”, and with the format() method, it will fill those placeholders with other strings.

# Example
sample1 = "this is a {}, and also a {}".format("pen", "weapon")
print(sample1)
 
#results
# this is a pen, and also a weapon 

this looks fine for now. But try to do one for the string that we pass into the db.execute() above. If we print the string, it will give you the result below

select * from some_table where filter = 'going to Sam's party'

Notice the extra single quote on the filter? This will cause error and of course opening a whole world of opportunity for SQL injection attack. With the single quote unescaped, the rest of the string can be injected with other commands that will cause serious maintenance headache.

-- example: imagine the replacer string is "bleh'; drop table some_table; insert into user values ('some new malicious users'); --"
-- your query will become
select * from some_table where filter = 'bleh'; drop table some_table; insert into user values ('some new malicious users'); --'
-- note that double dash (--) is used for commenting. So the extra single quote will be ignored.

So, why does code 2 is a better way of string replacements? Because it has built in special character escaping mechanism in which all strings that are passed thorough this method will remain as a string instead of becoming a malicious codes.

db.execute("select * from some_table where filter = %s", ("bleh'; drop table some_table; insert into user values ('some new malicious users'); --"))
the code above will produce sql below

 

select * from some_table where filter = E'bleh\'; drop table some_table; insert into user values (\'some new malicious users\'); --'
Standard