Big Data is big. The technology is now being used across all industries, from manufacturing to healthcare to even relatively low-tech retail and hospitality firms. The main technology behind Big Data, Hadoop is a framework that lets calculations on massive data sets take place on clustered nodes of inexpensive hardware, often in the cloud.
Big Corporate Commitments to Big Data
According to the research firm Gartner, more than 40 percent of companies they surveyed will invest in Hadoop development over the next two years. In the manufacturing industry, another survey showed big data was a priority for more than 80% of firms.
For many companies, one of the biggest stumbling blocks is a lack of familiarity with the technology and a lack of staff with the necessary experience. Because of this, developers with Hadoop skills are able to pull down big salaries – the average annual salary for Hadoop developers is more than $115,000.
Multiple Options for Companies Using Hadoop
Companies that sell big data products are trying to reduce the skills threshold in several ways. All vendors offer training, of course. Cloud providers including Amazon and Google offer Hadoop as a Service, letting businesses more easily spin up a Hadoop environment. These on-demand environments let companies dive right into the analysis that matters to them, rather than focusing on details like provisioning nodes and tuning clusters. Companies like Oracle provide pre-packaged analytics for specific industries.
Multiple Options for Developers with Hadoop Skills
All of that means that developers with Hadoop skills have lots of opportunity available to them, including working with a company implementing its own big data projects, a cloud vendor implementing big data environments on demand, and a packaged software vendor creating standard analytics reports.
Get Training in Big Data Skills
Developers who want to work with Big Data should get training in Hadoop, but that's not the only skill they need. Big Data depends on databases, and NoSQL is the chief database technology used. Although many Big Data developers will work with vendor analytics products, understanding data mining and statistical analysis is still necessary. Big Data developers should also have real familiarity with at least one of the vendor Hadoop as a Service offerings.
Currently, most big data opportunities are in geographic areas with large clusters of technology firms, like Silicon Valley, New York, and Seattle. As big data usage continues to spread, so will the need for its skills, meaning the opportunities will spread across the country.
It's common these days for IT teams to have team members in multiple locations around the world, whether to take advantage of specialized talent or cost factors. Technology helps these scattered teams communicate, but there are still challenges that come when co-workers aren't co-located. Here are four tips for managers to help their remote teams work effectively.
Projects always work more efficiently when there's a plan, and planning is even more critical with remote staff. There are fewer opportunities for casual interactions and questions to clarify assignments, and if confusion crosses time zones, delays can extend for days. Make sure you have a plan, so everyone knows what they're expected to do and when it needs to be done.
Schedule time to communicate
Because team members don't see you in person on a regular basis, they don't often get a lot of feedback. Don't rely on email; it's not dynamic enough, and meaning doesn't always come through. Plan a regular virtual meeting, perhaps once a month, to meet with your remote staff and give development guidance and other feedback. When possible, use videoconferencing, not just audio, so facial expressions and other non-verbal feedback are part of the communication process.
Build processes and systems to support the team
When people are in the same place, you may not need formal processes to address issues that arise; casual communication and spur-of-the-moment working sessions help sort things out. When people are around the world, a formally defined process ensures that everyone knows how to raise concerns, and that everyone is able to contribute input to solutions.
Build team spirit
Remote teams still want to feel like part of the team. Make sure remote staff are included in team celebrations. If possible, have managers visit the remote site periodically, and bring senior members of the remote staff for working visits to the home office. Besides providing an opportunity to build a shared work culture, these out-of-office experiences allow you to get to know remote staff as individuals and treat them as the unique people they are.
Moving from a hands-on role to a manager's role means a big change in what you do on a daily basis. It also means a big change in how you relate to co-workers, especially if you now manage former peers. Here are three tips to help you adapt and succeed in this new role.
Draw on Your Experience, but Continue to Learn
The most important thing to recognize as a new manager is that you don't actually know how to do everything the job requires. Your technical skills will help with some aspects of the job, like developing project schedules and deciding if an application is ready to release, but the management role requires other skills, like budgeting and conducting performance reviews, that may not have been needed previously. Plan to take the necessary training and find mentors to help you continue to develop.
As an individual contributor, your success was evaluated solely based on your own performance. As a manager, your success depends on the success of your team. It's important to get team members to buy in to project priorities and deadlines, which means setting goals clearly and being open to feedback. Make sure the team knows you're open to their opinions by having an open-door policy. Some team members may hesitate to speak out in a group setting, so seek them out for one-to-one discussions.
When things aren't going well, get the team's perception of the problem and their input on ways to improve it, rather than dictating a solution or imposing a new process on them. Be sure to celebrate the team's success, too; you don't want them conditioned to expect bad news when you walk into the room.
Support Other's Development
Your success as a manager may inspire others to aspire to management roles. Encourage team members to grow and develop skills, technical and other. Take the annual goal-setting process seriously, and help team members set goals that are achievable, and will benefit them as well as the company. Create an environment that supports learning, by encouraging training. Mentor team members to help them develop. Helping your team develop their skills will make them stronger contributors, increase your team's success, and help you climb the management ladder.
You may think the transition of an application from development to production happens at the end of the software lifecycle, but achieving a successful transition requires thinking about it all the way back at the start of the development process. Here are 10 things you can do during development to make it easier for the operations team to take responsibility for the app in production:
1. Make it easy to gather metrics. A key part of the operations team's job is monitoring what the application is doing and how it's doing it. That's a lot easier when the application gathers basic information and provides hooks for ops to check other performance numbers.
2. Document system dependencies. No system runs in isolation, but if the ops team doesn't know your application depends on some obscure package, that package may end up getting deleted to free up space. Put together a list of all the necessary components to make sure they get installed and stay installed to keep your app running.
3. Degrade gracefully. Don't write an application that falls apart when there's any sort of problem. Applications should flag problems and alert the support team, but continue functioning as fully as possible. This means applications shouldn't fail to come up just because the log file system is full.
4. Keep compatibility in mind. If your application can run with past and future versions of third-party products, the DevOps team will have a much easier time deploying your application and managing it, especially if it runs in a shared environment where other applications may want different versions of those components.
5. Make it easy to change configuration settings. A key responsibility of the DevOps team is keeping applications running when the environment around them is shifting. Building connection strings and other configuration settings into compiled code makes it impossible for DevOps to do that.
6. Provide configurations settings and flags for as much as possible. If a feature isn't configurable or can't be turned off and on, the DevOps team has no way to control it and manage the impact in production.
7. Build with scalability in mind. If you're lucky, your application will be wildly successful and have to grow to support more users. A well-designed application will let DevOps handle this by running the app on more nodes. If you don't design the application well, it may need to go back to the development team for a rewrite before the user base can expand.
8. Think about support scripts. The support team would love to automate as much of their job as possible. This means making sure application functionality can be invoked from a command line, rather than only by clicks within a GUI.
9. Provide a regression test suite. DevOps wants the application to keep running when the environment around it changes. A robust regression test suite lets them verify the system will still work after they make changes.
10. Document. Document. Document. There's a ton of critical information that exists only in the developers' heads. Writing it down means the DevOps team won't need to call developers in the middle of the night to find out where to change a configuration setting.
Many roles in IT are, by nature, solitary work. But as user experience becomes more important, it is vital that your IT team be able to engage effectively with both internal and external consumers. Introverts may not choose to become more extroverted on their own. You may need to provide your team with both the encouragement and tools they need to increase their interactions with co-workers and customers. Consider these tips to help your introverted IT people engage more comfortably.
Expand their comfort zones
Many introverts who find personal interaction uncomfortable are quite chatty and engaged online. Continue to provide channels for continued online interaction, but also schedule some one-on-one and small-group meetings to slowly get them accustomed to increased engagement. Provide a mix of group and solo time for best results. Even those who make the leap to becoming more extroverted will likely need some alone time to recharge.
Bolster team engagement
Schedule team-building activities for your IT team to give them the opportunity to interact with employees in other departments and get to know them on an informal basis. Focus especially on group activities that require different competencies, work styles and cooperative efforts to succeed. The skills and relationships forged in team activities can be brought into the workplace to increase confidence and comfort on the job.
Share what they know
As with most people, introverts are more confident when speaking on topics they understand in depth. Begin a program for IT professionals to give presentations on their areas of expertise. Start with small groups or internal lunch-and-learns. Allow attendees to ask clarifying questions as needed, and solicit feedback on what they have learned.
Once the speaker has gained confidence in this controlled environment, consider larger venues for the most successful speakers. This can help position the employee and your organization as thought leaders.
As technology grows increasingly consumer-centric, it's critical that every employee is able to interact at every level. Change the internal perception of techies as being standoffish and introverted by providing opportunities for interaction. With increased visibility comes increased engagement, which is good for both the employee and the organization.
As the digital landscape becomes increasingly crowded across every channel, and users continue to tune out traditional advertising, it’s more challenging than ever to differentiate online. For this reason, more companies are seeking IT pros who are able to provide exceptional user experiences.
What makes a great user experience? Here are 10 important considerations for making your websites, apps, or programs user-friendly — and more likely to succeed.
The majority of companies still use Windows tools and operating systems for one primary reason: It’s what they’re used to. Familiarity is a key component for a successful user experience. Basically, it means that accomplishing something within the environment should be obvious and not require explanation, such as a back button — a familiar tool that’s used in every Internet browser.
2. Responsive feedback
Websites, programs, and applications often include a number of micro-tasks, such as login screens. These tasks should include validation through feedback whenever possible — such as notifying users when they’ve successfully logged in. Without relevant and responsive feedback, users end up focusing intensely on micro-tasks and become frustrated.
3. Smooth performance
It goes without saying that performance is a crucial issue for user experience. If a website, program, or app suffers from performance issues, the user perceives the product as poorly designed or malfunctioning, and won’t be likely to continue using that product.
This attribute relates to the level or degree to which the use of an application, website, or program is obvious to the user. In addition to an intuitive interface, efficiency in features and functionality can enhance the user experience — particularly if there are more advanced tools that can be used with greater efficiency as the user becomes more familiar with the product.
Products that are actually helpful to users in accomplishing real goals deliver a better user experience. If a program, website, or application solves a business problem but disregards user needs, the experience is diminished for the user.
6. Relevant content delivery
A satisfying user experience should include relevant and valuable content, delivered in a timely manner. Ecommerce giant Amazon has mastered this aspect of the user experience with features like product recommendations, customer reviews, and a powerful and intuitive search function that deliver the right content at the right time.
7. Internal consistency
The user experience is enhanced when an interface or application handles similar tasks in similar ways, making the overall experience more intuitive and shortening the learning curve. In addition to internally consistent functionality, consistence in visual design is vital for presenting a professional and well-organized product.
8. External consistency
This refers to the visual appearance of a program, website, application, or interface aligning with its purpose and matching the expectations of its target audience — such as a polished and professional look for a website offering legal services, versus a fun and colorful theme for a site offering products for children.
9. Contextually appropriate
The interface for the user experience should match the environment in which the product will be used. For example, a product used for military applications should be more compact and rugged than one used in a restaurant, where the environment allows for a larger and more detailed interface.
In many cases, there is an implicit trust when users first work with an application or interface that the product will work as intended. Any issues that impact user engagement, such as error pages or non-working features, can erode that trust and diminish the user experience.
Security has been a concern for IT from the beginning, but with the increasing prevalence of mobile and the cloud — along with several high-profile data breaches, many of which have occurred in the past year at large organizations — keeping digital data secure is taking top priority for many. Recent research has found that security and data loss, the mobile workforce, and cloud technologies are the most important concerns for IT professionals in the coming year, as well as long-term over the next three-to-five years.
IT concerns by the numbers
The most recent annual Digital Leaders survey from BCS, The Chartered Institute for IT, found that the top three concerns for senior IT in the next 12 months are:
- Information security (60 percent)
- Cloud computing (55 percent)
- Mobile computing, including BYOD (53 percent)
For the next three-to-five years, the survey found that information security and cloud computing will remain priorities at 54 and 40 percent, respectively. Another long-term top concern is big data, which 42 percent of IT leaders said their companies would focus on during this time frame.
Another new study from IBM looked at IT security leaders specifically, and found the same chief concerns among them. This study found that:
- Almost 90 percent of IT security leaders are planning cloud initiatives or have already adopted the cloud
- 75 percent expect cloud security budgets to either increase or increase dramatically in the next three-to-five years
- Nearly 80 percent believe that potential security risks due to standards and regulations have increased in the past three years
- More than 70 percent stated real-time security intelligence is increasingly important for their companies
- Almost half said new security technologies are the top focus for their organizations
Challenges facing IT security
While more organizations have realized that developments in security, cloud, and mobile technology offer advantages in productivity, business continuity, and more, many IT leaders feel their companies are unable to meet the right goals. In the BCS survey, 92 percent of respondents felt their organizations lacked the resources to address the issues they’ve prioritized — with 53 percent stating a lack of enhanced IT skills among their existing workforce, and an equal number citing a shortage of additional IT staff.
Challenges discussed by the IBM survey primarily addressed mobile technology. In the study, less than half (45 percent) of IT security leaders believed they had an effective mobile device management (MDM) strategy in place. And while the majority of respondents said their companies were concerned about digital security going forward, most did not prioritize security for mobile devices.
As the movement toward mobile, the cloud, and the Internet of Things gains momentum, the IT of the future will have to prioritize security, and find new ways to work within diversified infrastructures to keep data safe.
With the rapid evolution of the Cloud, today’s companies no longer wonder whether they should climb aboard the Cloud bandwagon, because the answer is yes. The only question is how much of your infrastructure should move to the Cloud, and when.
Hybrid cloud environments—infrastructures that mix cloud services with on-premise solutions—are here to stay. But many organizations still aren’t leveraging the cloud in the most effective ways. The primary reason for this is a focus on costs, as companies try to replace on-premise solutions with the cheapest cloud services possible.
However, there’s far more value in choosing cloud components on the basis of innovation, and aligning your hybrid model with the primary functions of your business.
Identifying candidates for the cloud
When planning a hybrid cloud environment, CIOs need to separate their workloads by business function, and decide which functions would be best served by migrating to a cloud-based service. For most businesses, these functions will be the core enterprise workloads.
Infrastructure components like messaging, supply chain, HR, service management, Enterprise Resource Planning (ERP), and Customer Relationship Management (CRM) are typically ideal cloud candidates. These functions can be expensive to maintain through on-premise solutions, and difficult to integrate with a network. The cheaper, faster cloud alternatives for core enterprise workloads help organizations improve automation and cross-departmental functionality, delivering a more streamlined and cost-effective environment.
In these spaces, migrating to the cloud can be equated with buying innovation. Advances in software-as-a-service (SaaS), infrastructure-as-a-service (IaaS) and platform-as-a-service (PaaS) enable more features, better scalability, and decreased downtime for universal cloud-based services that are common to most businesses.
Choosing on-premise solutions
Where most companies err in developing hybrid solutions is choosing the cloud to save time and money for workloads that feed into core business functions. On-premise solutions allow for more robust and innovative platforms, which further key differentiators and help organizations remain competitive in their primary fields.
For functions that serve as your company’s main profit centers, on-premise solutions are the right choice. Your infrastructure investments should be directed toward these on-premise models—including all that money you’re saving by migrating core enterprise workloads to the cloud. On-premise solutions allow you to retain control of every aspect, which permits greater innovation and competitive advantage.
Blurring the line: Tips for maintaining hybrid environments
In most cases, dividing workloads into enterprise and core won’t always be a clean or simple process. The flow of information between departments must be taken into account, and full integration can be challenging with a hybrid model.
In an environment with cloud components, control is always decentralized to some degree. There will be parts of the IT process outside of your control, particularly when you’re feeding multiple cloud services into an on-premises environment. Within a hybrid system:
- The IT environment is chaotic (but it can be controlled)
- Operational performance will rely on external systems to some degree
- The user experience is highly distributed, with some aspects reliant on a third party
- Visibility is the key to optimized performance
Successful hybrid solutions will extend visibility to every component of the network layer. With orchestrated visibility, you can control the flow of information even when some elements are outside your control.
Choosing the right components to migrate to the cloud, and ensuring a seamless information flow with high visibility, will help you develop and maintain an effective hybrid environment that delivers optimal ROI for your organization.
Applications can only work with IT infrastructures behind them, but traditional IT roles have separated the people who work on hardware and software. Developers work on software, and operations teams keep the infrastructure humming. This separation of duties works when application cycles last weeks or months. But with the explosive growth of apps and software-as-a-service, the cycle has shortened to days, or even minutes — and the gap between developers and operations can pose serious problems.
Enter the DevOps movement. Powered by the proven idea that developers and operations are faster and more efficient when they work in tandem, DevOps involves a range of tools, techniques, and methods that bring the two factions together, helping them hit higher speeds across larger infrastructures.
What’s holding DevOps back?
While everyone wants the higher IT harmony, greater agility, and shorter time to market that DevOps can deliver, not everyone has been able to successfully implement this strategy — and it’s not because of the technology. A new Microsoft-sponsored study found that cultural barriers between developers and operations are the largest obstacle to any DevOps program.
The study found that while 54 percent of IT departments and companies are trying DevOps strategies on small projects, and 71 percent have pockets of automation, only 37 percent currently have formal DevOps solutions in place. The primary issue for over half of the survey respondents without formal strategies is “overcoming cultural habits inside my organization/company,” while a further 37 percent said they just don’t understand what DevOps entails.
Using existing tools in consistent ways
Mature tools already exist for implementing DevOps strategies in IT departments and companies of all sizes. Successful products like Puppet, GuardRail, Ansible, and Chef are already paving the way for a harmonious developer-operations environment. What IT teams must consider is the existing mindset and differences in culture, and solutions that will encourage widespread use of these tools across the organization.
Focusing on company-wide buy-in and cooperation is the way forward for DevOps. And while the study puts forth the idea that organizations using the Microsoft platform are better positioned to implement DevOps successfully — naturally, since the survey was sponsored by Microsoft — there is a salient point in that Microsoft tools and systems are built to work together. What’s more, the software giant has recently adopted a more developer-friendly stance, with more room for open source third-party integration.
At the heart of the issue, the study states that a successful DevOps environment will let developers and operations use the tools they want to use, on a more homogenous platform that allows for easier deployment across the board. Companies that focus on collaboration between people first, and technology second, will realize the DevOps dream of speed and efficiency sooner.
Need help implementing – or understanding – DevOps? Contact the IT experts at The Armada Group today and let our experienced staff share their knowledge and know how with you!
IT budgets can be tricky to handle. Ideally, you’d like to have things perfectly planned, so at the end of the year your budget balance is zero — but that happens rarely, if ever. Often you’ll have a little left over as the year comes to an end. So do you spend it wastefully, or leave it sitting there and risk having your budget cut next year because upper management decides you don’t need as much?
Fortunately, there are options for a middle ground. Here are some quick ideas on using minimal remaining IT budget funds to create maximum impact at the end of the year.
Invest in team mobility
Company-provided mobile devices can increase your team’s productivity, along with their satisfaction and loyalty. They also make a great holiday gift. If you have enough left in your budget, investing in top-of-the-line tablets or smartphones for your IT team is a smart move, helping to reward employees’ hard work while also making them more connected and productive.
Pop up your marketing
If you’re a smaller company or consulting firm, chances are your marketing is the lowest priority on your budget list. But conversely, your business can’t grow if you don’t invest in marketing. When you’re facing a year-end budget surplus, turning those funds into a marketing investment is one of the best gifts you can give your business — with a good chance of netting you an increased budget for next year.
Expand your cloud
Even if you’re not using the cloud much right now, you will be soon. The tech industry as a whole is on a rapid march toward cloud platforms and services, and as support for legacy solutions dwindles, the cloud is becoming a greater necessity.
If you’re already using cloud services on a limited basis, now is the perfect time to expand. Use your budget surplus to increase your cloud storage or bandwidth, or add more features to your existing platforms and services. The more agile your cloud, the better your business will perform in a cloud-based near future.
Invest in open source
One of the main attractions of open source platforms is that they’re free, or at least significantly lower in cost than popular enterprise solutions — so why would you invest money in them? Well, if you’re currently operating with proprietary solutions, it’s going to cost you to migrate to open source. But the benefits of moving to open source platforms are numerous, and the investment will pay for itself in the long run.
Open source has evolved from a complex toy to a major enterprise force, and many open source platforms are more reliable, more secure, and more robust than legacy solutions. Look into the options, and consider investing your surplus budget in an improved and more cost-effective platform.
Untangle your cable nests
Over time, server rooms can start to look like someone held a spaghetti party and forgot to clean up. If you have cabling strewn around the floor, and cracked or dried-out cable housing on some of your older cables, it’s time to upgrade — and a year-end budget surplus is the perfect excuse to handle this task.
Depending on your budget amount, you can either purchase replacement cables and have your team install them as needed, or hire a professional to come in and clean up in one pass.
Back up your backups
Being in IT means you already have a backup solution — but is it enough? Backup redundancy is often a small investment that can result in substantial returns. No backup is failsafe, and data loss can cause catastrophic consequences that often lead to business shutdown.
Investing in extra backup is never a waste of funds. At minimum, you should have both an online and a local backup solution in place, with scalable capabilities that can handle more than expected increases. If you already have this, you might consider a fireproof local backup solution or remote physical backup to guard against theft or natural disasters.
Still need more help coming up with solutions for a surplus end of year budget? Contact the IT pros at The Armada Group for more ideas and help implementing these changes!