Architectural design is concerned with understanding how a system should be organized and designing the overall structure of that system. Architectural design is the first stage in the software design process. It is the critical link between design and requirements engineering, as it identifies the main structural components in a system and the relationships between them. The output of the architectural design process is an architectural model that describes how the system is organized as a set of communicating components.
In agile processes, it is generally accepted that an early stage of the development process should be concerned with establishing on overall system architecture. Incremental development of architectures is not usually successful. While refactoring components in response to changes is usually relatively easy, refactoring a system architecture is likely to be expensive.
Picture 1 shows an abstract model of the architecture for a packing robot system that shows the components that have to be developed. This robotic system can pack different kinds of object. It uses a vision component to pick out objects on a conveyor, identify the type of object, and select the right kind of packaging. The system then moves objects from the delivery conveyor to be packaged. It places packaged objects on another conveyor. The architectural model shows these components and the links between them.
In practice, there is a significant overlap between the processes of requirements engineering and architectural design. Ideally, a system specification should not include any design information. This is unrealistic except for very small systems. Architectural decomposition is usually necessary to structure and organize the specification. Therefore, as part of the requirements engineering process, you might propose an abstract system architecture where you associate groups of system functions of features with large-scale components or sub-systems. You can then use this decomposition to discuss the requirements and features of the system with stakeholders.
You can design software architectures at two levels of abstraction, which I call architecture in the small and architecture in the large:
1. Architecture in the small is concerned with the architecture of individual programs. At this level, we are concerned with the way that an individual program is decomposed into components. This chapter is mostly concerned with program architectures.
2. Architecture in the large is concerned with the architecture of complex enterprise systems that include other systems, programs, and program components. These enterprise systems are distributed over different computers, which may be owned and managed by different companies.
Picture 1 The architecture of a packing robot control system
Software architecture is important because it affects the performance, robustness, distributability, and maintainability of system. Bosch (2000) discusses, individual components implement the functional system requirements. The non-functional requirements depend on the system architecture--the way in which these components are organized and communicate. In many system, non-functional requirements are also influenced by individual components, but there is no doubt that architecture of the systems is the dominant influence.
Bass, et al. (2003) discuss three advantages of explicitly designing and documenting software architecture :
1. Stakeholder communication. The architecture is a high-level presentation of the system that many be used as a focus for discussion by a range of different stakeholders.
2. System analysis. Making the system architecture explicit at an early stage in the system development requires some analysis. Architectural design decisions have a profound effect on whether or not system can meet critical requirements such as performance, reliability, and maintainability.
3. Large-scale reuse. A model of system architecture is a compact, manageable, description of how a system is organized and how the components interoperate. The system architecture is often the same for system with similar requirements and so can support large-scale software reuse.
Testing is intended to show that a program does what it is intended to do and to discover program defects before it it put into use. When we test software, we execute a program using artificial data. We check the result of the test run for errors, anomalies, or information about the program's non-functional attributes. The testing process how distinct goals:
1. To demonstrate to the developer and the customer that the software meets its requirements. For custom software, this means that there should be at least one test for every requirement in the requirements document. For generic software products, it means that there should be tests for all of the system features, plus combinations of these features, that will be incorporated in the product release.
2. To discover situations in which the behavior of the software is incorrect, undesirable or does not conform to its specifications. These are a consequence of software defects. Defect testing is concerned with rooting out undesirable system behavior such as system crashes, unwanted interactions with other systems, incorrect computations, and data corruption.
The first goal leads to validation testing, where you expect the system to perform correctly using a given set of test case that reflect the system's expected use. The second goal leads to defect testing, where the test cases are designed to expose defects. The test cases in defect testing can be deliberately obscure and need not reflect how the system is normally used. Of course, there is no definite boundary between these two approaches to testing. During validation testing, you will find defects in the system; during defect testing, some of the tests will show that the program meets its requirements.
Picture 1. An Input-Output Model of Program Testing
Let see Picture 1, it will help to explain the differences between validation testing and defect testing. Think of the system being tested as a black box. The system accepts inputs from some input set and generates outputs in an output set . Some of the outputs will be erroneous. These are the outputs in set
Let us get one thing straight. The only estimation I have done earlier is during my engineering days where I worked out quantities and rates for construction work. The key advantage then was that most of the design and specifications were finalized before the estimation began. If not, one could quote a vague figure called the 'accepted market rate', and then go on changing that figure [provided one had the skill set ;) to do so] as and when the design and specs were finalized.
For others with my kind of background, estimating a software project based upon a few discussions with the customer and stakeholders is nonsense. Estimating a software project is like estimating what it would take to invent something without knowing what that something is. Function point analysis and cocomo(?) is for geeks. Quite often, software estimates are way off the mark, and the end result being some geeks are then given pink slips.
And so, I did a sensible thing. I stayed well clear of estimating software projects. But situations change and it is getting more difficult to avoid this dumb task. The inability to estimate projects, resources, risks etc. also means inability to get perks, broaden our horizons, and rise up the geek ladder. Meanwhile, there are more mouths to feed, more financial commitments, more taxes to handle. And anyways, there is no point doing low level coding all your life. One should strive to reach a level where one can really screw things up.
Right then, enough of cribbing. It doesn’t pay. And my superiors are better at it. What I aim to do is to figure out a way to make estimation of a regular software project similar to estimating a regular engineering project. A typical engineering project estimate contains several parts as follows:
- Identifying the items of work.
- Fixing a unit for each item of work.
- Calculating the quantity of work.
- Calculating the rate for each item of work.
- Fine tuning the final cost.
The rate is the cost of resources involved per unit of work. The resources are usually:
- Tools and plants.
- Profit margin.
The overheads involve:
- The establishment charges.
- Depreciation of tools and plants.
- Interest factor etc.
- Licenses, permits, taxes.
The same concept can be extended for estimating software projects. We start by identifying and describing items of work, then the units for the item, the quantity, and so on. Describing items of work can be fairly irritating form a techie point of view, thanks to its verbose nature. The rate part should look normal enough. Usually, overheads and profit margin [decided by authorized persons only ;)] are taken as a percentage. By now, many would be smirking and/or condemning this kind of an approach. The main defense in favor of this approach is that most engineering works have to last for years, the project durations range from months to years, and there are many non technical people who believe they know things better than engineers (in some case, it's true). Many engineering projects have critics ranging from politicians, journalists, lawyers, social workers, citizens, idiots, and even engineers. Engineering projects include the Egyptian pyramids, space exploration, or manufacturing paper clips. So, why can't engineering techniques be applied to software development - a field that includes engineers of software, hardware, computers, and networking, and where reliability, robustness, and scalability are required.
Identifying items of work
Okay, now for the first part. What does an item of work consist of? In business applications, this could be a workflow, sub workflow, or development of components or sub components. We also need to classify the items of work (i.e., identify types of work).
In the following example, I will create some categories of work in software projects. A typical classification tree could be as shown in the figure below:
We can continue working out more categories and subcategories, but for now, we will attempt to estimate within the categories shown above. All the items under hardware category and sub categories are easier to estimate as we can get the costing from their purchase/market value. The same applies to the operating systems, run time software, and tools. The workstation cost includes establishment charges like office space, electricity, water, air conditioning etc. Other costs like renting, licenses, work permits, and taxes for acquiring each item should be added as applicable. Some overheads and hidden costs are not reflected in the tree. These are usually estimated and added as either a lump sum cost or as a percentage of the overall cost.
The actual software development costing starts from the category 'Application'. Most business applications have several modules. A module is a feature in an application. Now, we start to identify each item of work for each of its categories.
The category 'Framework' provides the backbone on which all the other components in the application link to or depend upon to do common tasks.
The framework development could include the following items of work:
- Data access components
- Exception handling
- Data Export
- Services (web services, remoting, DCOM, COM+, application server etc.)
- Security (authorization, authentication, code access, encryption)
- Resource Files
- Miscellaneous (utilities, constants, enumerations)
The category 'Presentation' could include the following items of work:
- GUI (Forms and controls)
- Custom controls
- Animation and Graphics
The category 'Business' could include the following items of work:
- Coding (for work flows and business requirements)
- Validating data
- Formulas and calculations
- Data manipulations
- Implementing business rules
The category 'Data formatter' is used to convert data into acceptable formats before sending and after receiving. The formats can be XML, value objects (an object that is used to carry data from tier to tier), and encryption.
The category 'Data storage' could have the following items of work:
- Database design
- Writing SQLs, procedures, triggers
- Performance tuning
- Flat Files (including text, XML, graphics, multimedia, etc.)
The Application itself is a category. In the category Application, an item of work is a work flow. A work flow itself may consists of other workflows and processes. The most common features when fixing a rate for an item of work are the expected amount of work to be done (GUI objects, lines of code, non private methods,) and expected complexity. This depends upon the experience of the bidding company, technology being used, and support available. The amount of work increases with the size of the workflow. If the workflow has a lot of rules, branches, sub workflows, and processes, it indicates complexity. Complex workflows should be broken down into simpler workflows. Therefore, the basis of estimation would be to develop as many workflows as possible and to put in the maximum number of details into the workflow. In case, developing all the workflows is not possible (often the case), then (the business analyst should be able to) map expected workflows to existing workflows so that estimation can be done with better accuracy. Trying to estimate a project without doing the workflows is practically impossible except in the case of static web pages.
Now that we have identified some of the items of work, we need to explain them in detail. Determine what exactly is going to be provided (and maybe even what is not being provided), the corresponding specifications, the cost of manpower, hardware, documents and other deliverables, overheads up to completion (or delivery). In software, this would also include the cost of testing, quality assurance, user acceptance testing, and warranties.
After describing an item of work, fix the unit for each item. For form development, the unit could be done as numbers (no.s). Note, the forms maybe further divided into categories depending upon the complexity and content in it. So, do not put all the forms into one single item of work unless all forms are similar to develop. In the Business category, we should identify the main business objects, components, or features, and categorize them depending upon the content and complexity. Again the unit for business category works could be no.s.
The database could be estimated based upon the number of database objects expected, and the complexity of the tables, views, SQLs, procedures, normalization, and level of data integrity required.
Once a workflow is done, it should be studied carefully to identify the items of work involved. Below are a set of workflows to request and provide a resource.
Let us study the resource allotment workflow for different categories.
Category Data Storage
This requires resource details to be stored so that resources can be easily stored and retrieved. This involves a relation database management system. The database should contain tables to store resources based upon certain characteristics, say human resources, items, and documents. To retrieve these faster, they can be further categorized and have keywords associated with them. All these categories would themselves be stored in another table(s) and mapped to their parent categories.
The request for resource should be mapped to the resources found. Therefore, all requests are stored in a table and can be categorized based upon priority, impact, department/person placing the request, and availability of resource.
Finally, the different criteria for accepting or rejecting a resource should be recorded along with the department/person who accepted or rejected the resource, along with an explanation. The criteria for accepting/rejecting a resource could be stored in another table and also could be categorized.
To place requests, search and allocate for resources would require writing SQLs. By viewing the functionality of the screens (if prototypes are available) as well as the workflow itself gives a fair idea of the complexity of the SQLs to be developed.
So right away, we get an idea of what is required from the data storage point of view.
The persons in charge of resource allocation would need to view the requests for resources. They should also be able to sort resources based upon the status of the request, date of the request, priority, impact, department, costing etc. This would help them decide how to handle each request. The resource in charge persons should be able to search for resources based upon categories and sub categories, keywords, status, location etc. Finally, a resource should be mapped to a request. Screens should be developed to display requests details, resource details, resources and requests mapping, search for requests and resources, etc. The display can be on WinForms, browser or console.
The Business category usually contains the interfaces and business objects. The business object may perform calculations (say, time period can be calculated from date(s) provided), implementing business rules (like whether the particular department can be given a certain resource), sequence of calling the business methods, and interactions with the data tier.
In the resource allocation workflow, we could have business objects such as requester, resource manager, resource, and search engine. Each object required can be considered while estimating the cost of the workflow.
Category Data Formatter
The Data formatter ensures that the format of the data is correct. If a particular data were of type float, then the data formatter would convert the data passed to it into a float before assigning it. Else it would not allow the data to be assigned at all. In case of XML, it would ensure that the XML document is built as per some predefined format. The data formatter could also be part of the framework. A data formatter can also be thought of as an object to carry data from one tier to another.
The framework components required by the resource allotment workflow could consist of tier to tier communication, security features, custom components, libraries, resource files, data tier access, error handling, and possibly logging, email, fax, and third party interaction. The extent to which each of these would be developed should be considered after looking at the entire application. But for our item of work, we should take a fraction of the framework development cost into account. This is because the framework is a development requirement and not a business requirement. So, only consider the features of the framework that would be required for the flow and add an approximate cost to the rate.
Item of Work
Let us call the item of work as 'resource allotment'. Now that we have looked at the item of work with respect to different categories, it will be easier to describe it. A critical aspect is the labor or man-hours involved to develop the workflow. Most companies have their own metrics on the development cost based upon efforts put into previous projects. Based upon the previous project metrics, they calculate the number of man-hours.
Now we put the item of work into an estimation table as shown below:
Developing and deploying the work flow resource allotment including the cost coding for sub workflows, components, display screens, inserting, modifying, deleting and retrieving data, passing data between client systems, servers, databases and external services, validating and formatting data, implementing business rules, including database design, application design, quality assurance, unit/integration/black box/load/user acceptance testing, hardware/software tools/operating-systems/compilers/labor/ licenses/permits/shipping/ deliverables.
For an inexperienced estimator, it would be difficult to fix the cost for the item of work shown in the table above. The best way is to describe the item of work with as many details as possible, and then try to arrive at the cost of each detail. Take the help of experienced persons (architects, managers, developers, testers, technical writers. etc.) to identify more details (including hardware, software, man hours, overheads) and their corresponding costs. Then simply add it up and compare the final figure against the cost of similar work done earlier.
Scrum is a subset of Agile and one of the most popular process frameworks for implementing Agile. It is an iterative software development model used to manage complex software and product development. Fixed-length iterations, called sprints lasting one to two weeks long, allow the team to ship software on a regular cadence. At the end of each sprint, stakeholders and team members meet to plan next steps.
Scrum follows a set of roles, responsibilities, and meetings that never change. For example, Scrum calls for four ceremonies that provide structure to each sprint: sprint planning, daily stand-up, sprint demo, and sprint retrospective. During each sprint, the team will use visual artifacts like task boards or burndown charts to show progress and receive incremental feedback.
Jeff Sutherland created the Scrum process in 1993, taking the term “Scrum” from an analogy in a 1986 study by Takeuchi and Nonaka published in the Harvard Business Review. In the study, Takeuchi and Nonaka compare high-performing, cross-functional teams to the Scrum formation used by Rugby teams. The original context for this was manufacturing, but Sutherland, along with John Scumniotales and Jeff McKenna, adapted the model for software development.
Advantages of Scrum
Scrum is a highly prescriptive framework with specific roles and ceremonies. While it can be a lot to learn, these rules have a lot of advantages. The benefits of Scrum include:
- More transparency and project visibility: With daily stand-up meetings, the whole team knows who is doing what, eliminating many misunderstandings and confusion. Issues are identified in advance, allowing the team to resolve them before they get out of hand.
- Increased team accountability: There is no project manager telling the Scrum Team what to do and when. Instead, the team collectively decides what work they can complete in each sprint. They all work together and help each other, improving collaboration and empowering each team member to be independent.
- Easy to accommodate changes: With short sprints and constant feedback, it’s easier to cope with and accommodate changes. For example, if the team discovers a new user story during one sprint, they can easily add that feature to the next sprint during the backlog refinement meeting.
- Increased cost savings: Constant communication ensures the team is aware of all issues and changes as soon as they arise, helping to lower expenses and increase quality. By coding and testing features in smaller chunks, there is continuous feedback and mistakes can be corrected early on, before they get too expensive to fix.
Disadvantages of Scrum
While Scrum offers some concrete benefits, it also has some downsides. Scrum requires a high level of experience and commitment from the team and projects can be at risk of scope creep.
Here are the disadvantages of Scrum:
- Risk of scope creep: Some Scrum projects can experience scope creep due to a lack of specific end date. With no completion date, stakeholders may be tempted to keep requesting additional functionality.
- Team requires experience and commitment: With defined roles and responsibilities, the team needs to be familiar with Scrum principles to succeed. Because there are no defined roles in the Scrum Team (everyone does everything), it requires team members with technical experience. The team also needs to commit to the daily Scrum meetings and to stay on the team for the duration of the project.
- The wrong Scrum Master can ruin everything: The Scrum Master is very different from a project manager. The Scrum Master does not have authority over the team; he or she needs to trust the team they are managing and never tell them what to do. If the Scrum Master tries to control the team, the project will fail.
- Poorly defined tasks can lead to inaccuracies: Project costs and timelines won’t be accurate if tasks are not well defined. If the initial goals are unclear, planning becomes difficult and sprints can take more time than originally estimated.
Roles in Scrum
There are three specific roles in Scrum. They are:
- Product Owner: The Scrum Product Owner has the vision of what he or she wants to build and conveys that vision to the team. The Product Owner focuses on business and market requirements, prioritizing all the work that needs to be done. He or she builds and manages the backlog, provides guidance on which features to ship next, and interacts with the team and other stakeholders to make sure everyone understands the items in the product backlog. The Product Owner is not a project manager. Instead of managing the status and progress, his or her job is to motivate the team with a goal and vision.
- Scrum Master: Often considered the coach for the team, the Scrum Master helps the team do their best possible work. This means organizing meetings, dealing with roadblocks and challenges, and working with the Product Owner to ensure the product backlog is ready for the next sprint. The Scrum Master also makes sure the team follows the Scrum process. He or she doesn’t have authority over the team members, but he or she does have authority over the process. For example, the Scrum Master can’t tell someone what to do, but could propose a new sprint cadence.
- Scrum Team: The Scrum Team is comprised of five to seven members. Everyone on the project works together, helps each other, and shares a deep sense of camaraderie. Unlike traditional development teams, there are not distinct roles like programmer, designer, or tester. Everyone completes the set of work together. The Scrum Team owns the plan for each sprint; they anticipate how much work they can complete in each iteration.
Steps in the Scrum Process
There are a specific, unchanging set of steps in the Scrum flow. They include:
- Product backlog: The Product Owner and Scrum Team meet to prioritize the items on the product backlog (the work on the product backlog comes from user stories and requirements). The product backlog is not a list of things to be completed, but rather it is a list of all the desired features for the product. The development team then pulls work from the product backlog to complete during each sprint.
- Sprint planning: Before each sprint, the Product Owner presents the top items on the backlog to the team in a sprint planning meeting. The team then chooses which work they can complete during the sprint and moves the work from the product backlog to the sprint backlog (which is a list of tasks to complete in the sprint).
- Backlog refinement/grooming: At the end of one sprint, the team and Product Owner meet to make sure the backlog is ready for the next sprint. The team may remove user stories that aren’t relevant, create new stories, reassess the priority of stories, or split user stories into smaller tasks. The purpose of this “grooming” meeting is to ensure the backlog only contains items that are relevant and detailed, and that meet the project’s objectives.
- Daily Scrum meetings: The Daily Scrum is a 15-minute stand-up meeting where each team member talks about their goals and any issues that have come up. The Daily Scrum happens every day during the sprint and helps keep the team on track.
- Sprint review meeting: At the end of each sprint, the team presents the work they have completed at a sprint review meeting. This meeting should feature a live demonstration, not a report or a PowerPoint presentation.
- Sprint retrospective meeting: Also at the end of each sprint, the team reflects on how well Scrum is working for them and talks about any changes that need to be made in the next sprint. The team may talk about what went well during the sprint, what went wrong, and what they could do differently.
Tools, Artifacts, and Methods in Scrum
In addition to roles and ceremonies, Scrum projects also include certain tools and artifacts. For example, the team uses a Scrum board to visualize the backlog or a burndown chart to show outstanding work. The most common artifacts and methods are:
- Scrum board: You can visualize your sprint backlog with a Scrum task board. The board can have different forms; it traditionally involves index cards, Post-It notes, or a whiteboard. The Scrum board is usually divided into three categories: to do, work in progress, and done. The Scrum Team needs to update the board throughout the entire sprint. For example, if someone comes up with a new task, she would write a new card and put it in the appropriate column.
- User stories: A user story describes a software feature from the customer’s perspective. It includes the type of user, what they want, and why they want it. These short stories follow a similar structure: as a <type of user>, I want to <perform some task> so that I can <achieve some goal.> The development team uses these stories to create code that will meet the requirements of the stories.
- Burndown chart: A burndown chart represents all outstanding work. The backlog is usually on the vertical axis, with time along the horizontal axis. The work remaining can be represented by story points, ideal days, team days, or other metrics. A burndown chart can warn the team if things aren’t going according to plan and helps to show the impact of decisions.
- Large-Scale Scrum (LeSS): If you want to scale elements of Scrum to hundreds of developers, the Large-Scale Scrum (LeSS) framework helps extend the rules and guidelines without losing the core of Scrum. The principles are taken directly from Scrum, however focuses on scaling up without adding additional overhead (like adding more roles, artifacts, or processes).
- Timeboxing: A timebox is a set period of time during which a team works towards completing a goal. Instead of letting a team work until the goal is reached, the timebox approach stops work when the time limit is reached. Time-boxed iterations are often used in Scrum and Extreme Programming.
- Icebox: Any user stories that are recorded but not moved to development are stored in the icebox.
The term “icebox” was created by Pivotal Tracker, an Agile project management tool.
- Scrum vs RUP: While both Scrum and Rational Unified Process (RUP) follow the Agile framework, RUP involves more formal definition of scope, major milestones, and specific dates (Scrum uses a project backlog instead of scope). In addition, RUP involves four major phases of the project lifecycle (inception, elaboration, construction, and transition), whereas Scrum dictates that the whole “traditional lifecycle” fits into one iteration.
- Lean vs Scrum: Scrum is a software development framework, while Lean helps optimize that process. Scrum’s primary goal is on the people, while Lean focuses on the process. They are both considered Agile techniques, however Lean introduces two major concepts: eliminating waste and improving flow.
How to Get Started with Scrum
Working with Scrum often means changing the team’s habits. They need to take more responsibility, increase the quality of the code, and boost speed of delivery. This level of commitment acts as a change agent; as the teams commit to sprint goals, they are more and more motivated to get better and faster to deliver a quality product.
A good place to start with Scrum is to talk about the roles. Every project must have a Scrum Master, Product Owner, and Scrum Team. You may want to talk about who should be the Scrum Master and Product Owner, or if these roles are already assigned, you may want to clarify their roles and responsibilities.
Depending on how familiar your team is with Scrum, you may also want to look into training sessions. Certified Scrum Coaches and Trainers and Scrum Alliance Registered Education Providers can help your team learn and embrace Scrum.
Our newest view, Card View, gives Agile teams a more highly-visual way to work, communicate, and collaborate in Smartsheet. Card View enables you to focus attention with rich cards, give perspective with flexible views, and prioritize and adjust work more visually. Display information on cards including custom fields, images, and color coding to better focus your team’s attention. Categorize cards into lanes to organize your work more visually.
See how easy it can be to use Smartsheet Card View during your next Scrum meeting.
Code Simplicity is a companion blog to author Max Kanat-Alexander’s application design book Code Simplicity: The Science of Software Development. Max is a software engineer at Google, and the chief architect of the Bugzilla Project – and his blog draws upon this experience to offer advice on simplifying software design. His mantra is ‘Complexity is stupid. Simplicity is smart’ – and after reading the blog, I’m inclined to agree.
In addition to being a former Microsoft program manager, Joel Spolsky is a co-founder of programming Q&A site StackExchange, the man behind software development company Fog Creek Software, and the awesome little browser-based workflow tool Trello. He’s been blogging since 2000, and his site is a goldmine of insight on software dev, management and business.
3) Scott Berkun
Scott Berkun’s eponymous blog is one of the most multi-faceted on this list, offering advice and insight into creativity, leadership and philosophy – alongside his experiences as a manager at giants Microsoft and WordPress. If you’re tired of reading the same old blog content, Scott’s blog offers a plethora of engaging info, all of which is designed to help you become a better person, as well as a better programmer.
Coding Horror is the outlet of seasoned web application developer (and, like Joel Spolsky above, co-founder of StackExchange) Jeff Atwood. The blog tackles all manner of software development and security topics, but it’s Jeff’s interest in the human component of development that makes the blog stand out. As Jeff himself says:
‘In the art of software development, studying code isn't enough; you have to study the people behind the software, too.’
Scott Hanselman’s blog tackles the full pantheon of software developer interests, covering technology, code, gadgets, dev culture and the web. As a former professor, and current employee of Microsoft, his hands-on advice is clear, concise and helpful. Unlike many of his contemporaries, Scott’s writing is also bursting with personality. If you’re a fan of Scott’s insight, you can also check out his three podcasts and YouTube channel.
Andy Hunt is a prolific author, a co-founder of the Agile Alliance, and part of the team that developed the Agile Manifesto. Andy’s blog tackles a diverse range of development topics, and unsurprisingly, offers some of the most interesting and unique insight into agile development anywhere on the web.
Paul Graham was one-half of the duo behind Viaweb, arguably the very first (started in 1995) software as a service company. Since then, he’s gone on to co-found Y Combinator, a start-up incubator that’s funded the likes of Dropbox, Reddit and Airbnb. Paul Graham’s Essays collates his long-form insights into developing SaaS businesses, and provides developers a wonderful insight into their role within the wider business world.
Federico is a professional mobile and web developer, and regularly blogs around coding (particularly PHP), software architecture and agile development. With a mixture of straight-to-the-point tutorials and, courtesy of his Twitter, a ton of tech news and insight, Frederico’s blog is a great read for any software developer.
10) David Walsh
Ponitkis is a blog of two halves, offering the latest in web technology, business and news, alongside a plethora of how-tos and guides. Author Christos Pontikis offers seriously in-depth instructions on all-manner of languages and frameworks, with his expert insights into PHP, jQuery and MySQL a serious incentive for any knowledge-hungry developers.
12) Six Revisions
13) Web Appers
WebAppers dedicates itself to sourcing and collating free open-source tools and resources, with the professional web dev and web designer in mind. In addition to a pantheon of almost 700 plugins, the blog shares a ton of actionable guidance and helpful advice, with a view to helping web developers use the tools in the most beneficial way possible.
Since its inception in 2005, ProgrammableWeb has been at the forefront of the evolving API economy. It offers a staggering amount of hands-on content, and manages to maintain its quality across an incredible publication schedule ranging as high as 10 posts per day. In addition to its fantastic blog content, ProgrammableWeb has a huge directory of APIs for web and mobile development, and a plethora of whitepapers and research.
16) Martin Fowler
Software developer Martin Fowler is a prolific author (having penned no less than seven programming books), and an even more prolific blogger. He writes primarily around agile, refactoring and project delivery – with a particular focus on the design of software systems, and ways to maximise the productivity of development. Whilst the blog is a great resource for all types of developer, it should have a special interest to those managing a development team.
17) Eric Sink
Eric Sink is a software developer at SourceGear – but prior to his current role, he served as project lead for the browser development team that prototyped a little-known browser called ‘Internet Explorer’. Since then, Eric has been blogging consistently around software development, with his advice, news roundups and opinions stretching all the way back to 2001.
18) The Daily WTF
If you’re looking to break-up the monotony of personal development, The Daily WTF should provide ample relief. The site pairs genuinely helpful development insights with an awesome sense of humour, creating a blog that’s as funny to read as it is useful. The site has a particular focus on how-not-to-guides, and the disastrous development stories its shares will easily consume your lunch break.
19) UIE Brainsparks
User Interface Engineering is a research and training company focused on web and application usability. Its Brainsparks blog is an industry-leading resource, covering all aspects of UI and UX development – with founder Jared Spool offering his expert insight on a weekly basis. In addition to the blog, UIE offer podcasts, long-form articles, event and seminars for devs interested in improving their UI skills.
Programmer turned publisher Dave Thomas blogs and tweets about all manner of development news and advice. Alongside tutorials, guides and opinions, Dave has developed his own Zen-like approach to the art of coding – creating the martial arts inspired CodeKata to help developers change their attitude to coding, and develop an always-learning mindset.
Open development breaks the data center down into its lowest-level components, which fit together by open standards. Still, with less than 2% of enterprise applications designed for horizontal scaling, enterprise IT should avoid lifting legacy apps onto open infrastructure.
Instead, put new workloads on building-block infrastructure, and renegotiate your hardware contracts to get ready for more open-standard hardware and software.
he problem, however, is IT administrators love scripts. They love creating the best scripts, fiddling with scripts that come from colleagues, and leaving little documentation when they move on to another job. IT automation must evolve from scripting to deterministic (defined workloads for tasks) then to heuristic design (automation based on data fed in operations). There are banks today that use heuristic automation because they have all the hardware that you could want, Govekar said. But they lack the ability to automatically place workloads that best at any given moment.
the control plane is abstracted from the hardware, and it's going on with every piece of equipment a data center can buy. Software-defined servers are established, software-defined networking is maturing and software-defined storage won't have much impact until at least 2017, Govekar said.
Don't approach software-defined everything as a cost saving venture, because the real point is agility. Avoid vendor lock-in in this turbulent vendor space, and look for interoperable application programming interfaces that enable data-center-wide abstraction. Also, keep in mind that the legacy data center won't die without a fight.
Big data analysis is used in a number of ways to solve problems today. For example, police departments reduce crime without blanketing the city with patrol cars, by pinpointing likely crime hot spots at a given point in time based on real-time and historical data.
Build new data architectures to handle unstructured data and real-time input, which are disruptive changes today. The biggest inhibitor to enterprise IT adoption of big data analytics, however, isn't the data architecture; it's a lack of big data skills.
Internet of Everything
is IT in charge of the coffee pot? If it has an IP address and connects to the network, it might be.
Internet-connected device proliferation combined with big data analytics means that businesses can automate and refine their operations. It also means security takes on a whole new range of end points. In data center capacity management, Internet of Everything means demand shaping and customer priority tiering, rather than simply buying more hardware.
Build a data center that can change, don't build to last, Govekar said.
For better or worse, business leaders want to know why you can't do what Google, Facebook and Amazon do.
Conventional hardware and software are not built for webscale IT, which means this trend relies on software-defined everything and open philosophies like the Open Compute Project. It also relies on a major attitude adjustment in IT where experimentation and failure are allowed.
our workforce is mobile. Your company's customers are mobile. Bring your own device has morphed into bring your own toys. The IT service desk can't fall behind this trend and risk giving IT a reputation of being out of touch.
Bring data segregation -- personal and business data and applications isolated from each other on the same device -- onto your technology roadmap now.
No one's congratulating IT on keeping the lights on and the servers humming, no matter how difficult it can be. Bimodal IT means maintaining traditional IT practices while simultaneously introducing innovative new processes -- safely.
Take the pace layering concept from application development and apply it to IT's roadmap, and find ways to get close to customers. Bimodal IT will make your team more diverse.
Business value dashboards
By 2017, the majority of infrastructure and operations teams will use dashboards to communicate with the outside world. Govekar made the analogy of the business-value dashboard vs. IT metrics to cruise ship reviews vs. cruise ship boiler calibration reports. They serve different purposes.
All the trends above feed shadow IT, where the business units steer around IT to gain agility.
Some IT teams are trying a new approach; rather than quash all shadow IT operations they find, these companies allow business users to set up shadow IT for projects and track the performance like a proof-of-concept trial. If the deployment succeeds, IT formally folds shadow IT into the organization.
- Definitions ?
DevOps is a new term emerging from the collision of two major related trends. The first was also called “agile system administration” or “agile operations”; it sprang from applying newer Agile and Lean approaches to operations work. The second is a much expanded understanding of the value of collaboration between development and operations staff throughout all stages of the development lifecycle when creating and operating a service, and how important operations has become in our increasingly service-oriented world
One definition Jez Humble explained to me is that DevOps is “a cross-disciplinary community of practice dedicated to the study of building, evolving and operating rapidly-changing resilient systems at scale.”
That’s good and meaty, but it may be a little too esoteric and specific to Internet startup types. I believe that you can define DevOps more practically as DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.
DevOps means a lot of different things to different people because the discussion around it covers a lot of ground. People talk about DevOps being “developer and operations collaboration,” or it’s “treating your code as infrastructure,” or it’s “using automation,” or “using kanban,” or “a toolchain approach,” or “culture,” or a variety of seemingly loosely related items. The best way to define it in depth is to use a parallel method to the definition of a similarly complex term, agile development. Agile development, according to Wikipedia and the agile manifesto, consists of four different “levels” of concern. I’ve added a fifth, the tooling level – talk about agile and devops can get way too obsessed with tools, but pretending they don’t exist is also unhelpful.
- Agile Values – Top level philosophy, usually agreed to be embodied in the Agile Manifesto. These are the core values that inform agile.
- Agile Principles – Generally agreed upon strategic approaches that support these values. The Agile Manifesto cites a dozen of these more specific principles. You don’t have to buy into all of them to be Agile, but if you don’t subscribe to many of them, you’re probably doing something else.
- Agile Methods – More specific process implementations of the principles. XP, Scrum, your own homebrew process – this is where the philosophy gives way to operational playbooks of “how we intend to do this in real life.” None of them are mandatory, just possible implementations.
- Agile Practices – highly specific tactical techniques that tend to be used in conjunction with agile implementations. None are required to be agile but many agile implementations have seen value from adopting them. Standups, planning poker, backlogs, CI, all the specific artifacts a developer uses to perform their work.
- Agile Tools – Specific technical implementations of these practices used by teams to facilitate doing their work according to these methods. JIRA Agile (aka Greenhopper), , et al.
the different parts of DevOps that people are talking about map directly to these same levels.
- DevOps Values – I believe the fundamental DevOps values are effectively captured in the Agile Manifesto – with perhaps one slight emendation to focus on the overall service or software fully delivered to the customer instead of simply “working software.” Some previous definitions of DevOps, like Alex Honor’s “People over Process over Tools,” echo basic Agile Manifesto statements and urge dev+ops collaboration.
- DevOps Principles – There is not a single agreed upon list, but there are several widely accepted attempts – here’s John Willis coining “CAMS” and here’s James Turnbull giving his own definition at this level. “Infrastructure as code” is a commonly cited DevOps principle. I’ve made a cut at “DevOps’ing” the existing Agile manifesto and principles here. I personally believe that DevOps at the conceptual level is mainly just the widening of Agile’s principles to include systems and operations instead of stopping its concerns at code checkin.
- DevOps Methods – Some of the methods here are the same; you can use Scrum with operations, Kanban with operations, etc. (although usually with more focus on integrating ops with dev, QA, and product in the product teams). There are some more distinct methods, like Visible Ops-style change control and using the Incident Command System for incident reponse. The set of these methodologies are growing; a more thoughtful approach to monitoring is an area where common methodologies haven’t been well defined, for example.
- DevOps Practices –Specific techniques used as part of implementing the above concepts and processes. Continuous integration and continuous deployment, “Give your developers a pager and put them on call,” using configuration management, metrics and monitoring schemes, a toolchain approach to tooling… Even using virtualization and cloud computing is a common practice used to accelerate change in the modern infrastructure world.
- DevOps Tools – Tools you’d use in the commission of these principles. In the DevOps world there’s been an explosion of tools in release (jenkins, travis, teamcity), configuration management (puppet, chef, ansible, cfengine), orchestration (zookeeper, noah, mesos), monitoring, virtualization and containerization (AWS, OpenStack, vagrant, docker) and many more. While, as with Agile, it’s incorrect to say a tool is “a DevOps tool” in the sense that it will magically bring you DevOps, there are certainly specific tools being developed with the express goal of facilitating the above principles, methods, and practices, and a holistic understanding of DevOps should incorporate this layer.
Mark Zukerberg's challenge for 2016 was to build a simple AI to run his home -- like Jarvis in Iron Man. His goal was to learn about the state of artificial intelligence -- where we're further along than people realize and where we're still a long ways off. These challenges always lead me to learn more than he expected, and this one also gave me a better sense of all the internal technology Facebook engineers get to use, as well as a thorough overview of home automation.
So far this year, he has built a simple AI that he can talk to on his phone and computer, that can control his home, including lights, temperature, appliances, music and security, that learns his tastes and patterns, that can learn new words and concepts, and that can even entertain Max. It uses several artificial intelligence techniques, including natural language processing, speech recognition, face recognition, and reinforcement learning, written in Python, PHP and Objective C. In this note, he'll explain what he built and what he learned along the way.
- Getting Started: Connecting the Home
In some ways, this challenge was easier than he expected. In fact, my running challenge (I also set out to run 365 miles in 2016) took more total time. But one aspect that was much more complicated than he expected was simply connecting and communicating with all of the different systems in my home.
Before he could build any AI, he first needed to write code to connect these systems, which all speak different languages and protocols. We use a Crestron system with our lights, thermostat and doors, a Sonos system with Spotify for music, a Samsung TV, a Nest cam for Max, and of course my work is connected to Facebook's systems. he had to reverse engineer APIs for some of these to even get to the point where he could issue a command from my computer to turn the lights on or get a song to play.
Further, most appliances aren't even connected to the internet yet. It's possible to control some of these using internet-connected power switches that let you turn the power on and off remotely. But often that isn't enough. For example, one thing he learned is it's hard to find a toaster that will let you push the bread down while it's powered off so you can automatically start toasting when the power goes on. he ended up finding an old toaster from the 1950s and rigging it up with a connected switch. Similarly, he found that connecting a food dispenser for Beast or a grey t-shirt cannon would require hardware modifications to work.
For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other
- natural language
This was a two step process: first he made it so he could communicate using text messages, and later he added the ability to speak and have it translate my speech into text for it to read.
It started simple by looking for keywords, like "bedroom", "lights", and "on" to determine he was telling it to turn the lights on in the bedroom. It quickly became clear that it needed to learn synonyms, like that "family room" and "living room" mean the same thing in our home. This meant building a way to teach it new words and concepts.
- Vision and Face Recognition
Even though he think text will be more important for communicating with AIs than people realize, he still think voice will play a very important role too. The most useful aspect of voice is that it's very fast. You don't need to take out your phone, open an app, and start typing -- you just speak.
To enable voice for Jarvis, he needed to build a dedicated Jarvis app that could listen continuously to what he say. The Messenger bot is great for many things, but the friction for using speech is way too much. My dedicated Jarvis app lets me put my phone on a desk and just have it listen. he could also put a number of phones with the Jarvis app around my home so he could talk to Jarvis in any room. That seems similar to Amazon's vision with Echo, but in my experience, it's surprising how frequently he want to communicate with Jarvis when I'm not home, so having the phone be the primary interface rather than a home device seems critical.
- Facebook Engineering Environtment
As the CEO of Facebook, he don't get much time to write code in our internal environment. he has never stopped coding, but these days he mostly build personal projects like Jarvis. he expected he'd learn a lot about the state of AI this year, but he didn't realize he would also learn so much about what it's like to be an engineer at Facebook. And it's impressive.
My experience of ramping up in the Facebook codebase is probably pretty similar to what most new engineers here go through. he was consistently impressed by how well organized our code is, and how easy it was to find what you're looking for -- whether it's related to face recognition, speech recognition, the Messenger Bot Framework [messenger.com/platform] or iOS development. The open source Nuclide [github.com/facebook/nuclide] packages he ve built to work with GitHub's Atom make development much easier. The Buck [buckbuild.com] build system he ve developed to build large projects quickly also saved me a lot of time. Our open source FastText [github.com/facebookresearch/fastTex...] AI text classification tool is also a good one to check out, and if you're interested in AI development, the whole Facebook Research [github.com/facebookresearch] GitHub repo is worth taking a look at.
One of our values is "move fast". That means you should be able to come here and build an app faster than you can anywhere else, including on your own. You should be able to come here and use our infra and AI tools to build things it would take you a long time to build on your own. Building internal tools that make engineering more efficient is important to any technology company, but this is something we take especially seriously. So he want to give a shout out to everyone on our infra and tools teams that make this so good.
Perbedaan QA engineer dan QA tester
Pada beberapa perushaan startup kita kerap kali mendengar istilah QA engineer dan QA tester. Kedua jabatan tersebut sangat penting dalam menjaga kualitas perangkat lunak. Apakah perbedaan antara QA engineer dan QA tester ?
QA engineer lebih fokus pada bagaimana proses pengembangan perangkat lunak dilakukan dengan baik daripada bagaimana hasil akhir perangkat lunak yang dihasilkan. Tugas QA engineer memastikan setiap proses pengembangan perangkat lunak berjalan dengan baik guna menjaga kualitas pengembangan perangkat lunak. Menjaga kualitas proses programming dan build/test menjadi salah satu fokus utama pada pekerjaan QA engineer.
Beberapa tugas QA engineer adalah
- Function – Test planning, Test Design and execution.
- Prepare test plans, develop test cases and execute tests with a focus on coverage.
- Engineers Quality. Good to answer – what is the quality of the product?
- Logical thinkers. Ability to resolve issues using abstraction. Capability in analysis/predictions/improvements.
- Can reconcile conflicting constraints.
- Process and metrics/measurement driven.
- Cost sensitive
- Good for System/functionality testing
- Best to have them involved in complete SDLC Cycle.
Berbeda dengan QA engineer, QA tester lebih berfokus kepada kualitas perangkat lunak yang digunakan. Pengujian-pengujian yang dilakukan oleh QA tester lebih fokus pada metode black-box. QA tester bertugas memastikan perangkat lunak yang di hasilkan sudah sesuai dengan keinginan user.
Beberapa tugas QA tester antara lain
- Function – Strong in test execution.
- Write and execute test cases – may not be coverage driven. Requirements driven testing.
- Determines Quality. Good to answer – did you find any bugs?
- Linear thinkers, low capability of analysis and re-usability of efforts/resources.
- Requires a defined environment. Typically are weak in finding solutions in ambiguous/constrained environments.
- Low process oriented capability.
- May not be cost sensitive (time, effort, monetary, etc.)
- Good for UI Testing
- Typical involvement is in later stage of SDLC
Kesimpulan yang bisa kita ambil, untuk menjaga kualitas perangkat lunak maka dibutuhkan proses QA pada proses pembangunan perangkat lunak/SDLC dan setelah perangkat lunak jadi dan siap rilis. Untuk memenuhi kebutuhan tersebut jabatan QA dibagi menjadi QA engineer dan QA tester. Dimana QA engineer fokus pada proses SDLC dan QA tester fokus pada proses setelah SDLC atau sebelum delivery ke user.
karena QA engineer melakukan test pada setiap proses SDLC dibandingkan dengan QA tester yang hanya berfokus pada testing aplikasi yang sudah jadi. QA engineer memiliki beban kerja yang lebih berat, sehingga salary pun lebih tinggi dari QA tester.
Jadi mau pilih yang mana?, QA engineer atau QA tester?
Software Framework adalah sebuah abstraksi yang digunakan untuk mempermudah pembuatan perangkat lunak. Software Framework menyamakan persepsi setiap pengembang pada abstraksi yang sama. Penyamaan persepsi ini mempermudah pengembang baru untuk mengetahui keseluruhan perangkat lunak dengan lebih cepat. Paradigma yang paling populer digunakan pada software framework adalah MVC. MVC memisahkan aplikasi menjadi 3 bagian fungsionalitsa yaitu model,view, dan controller.
Beberapa software framework yang sering digunakan adalah
- Laravel : full-stack web framework menggunakan paradigma MVC, memiliki fitur yang sangat lengkap dan pemodelan ORM(Object Relational Mapping) pada database
- CodeIgniter: full-stack web framework menggunakan paradigma MVC, memiliki fitur yang umum dan berjalan cepat dibandingkan framework php lainnya
- Lumen : Rest Api web framework dari Laravel. Lumen adalah versi REST API dari Laravel,lumen memiliki fitur yang lebih simpel dibandingkan Laravel
- Yii : full-stack web framework menggunakan paradigma MVC memiliki fitur lengkap dan beberapa scaffolding.
- Flask : minimal web framework, digunakan untuk membangun aplikasi web atau API
- Djanggo : full-stack web framework dengan paradigma MVC digunakan untuk membuat aplikasi web yang utuh
- Ruby on Rails: Pesaing laravel, memiliki kelengkapan fitur seperti laravel dan berjalan menggunakan bahasa ruby, memiliki kelebihan pada syntax nya yang sederhana.