Tag Archive for Big data

| August 13, 2013
Comments Off on Data Growth Tests Storage Capabilities

Data Growth Tests Storage Capabilities

 Data Growth Tests Storage CapabilitiesData Center Knowledge had an article by Steven Rodin, CEO of Storagepipe Solutions, that lays out the challenges that those of us charged with managing backups face every day. Storagepipe Solutions, which has been a provider of online backup services for business since 2001, has identified several emerging storage trends that organizations will need to overcome in the future.

Storagepipe Solutions logoIn the early days, the author says, organizations were primarily concerned with data protection, encryption and automation. The era of “Big Data” has changed those demands. The new demands are overwhelming most backup and storage systems. The article cites data from IBM (IBM), which claims that worldwide annual data production has actually exceeded worldwide storage capacity. Big Blue believes that demand for storage capacity is growing nearly 60 percent a year. The gap between the data that organizations produce and their ability to store it will continue to grow for years.

The Storagepipe Solutions CEO identified a number of important storage trends which are accelerating the growth rate of corporate data.  He provided a few of the most important factors.

Cheaper Storage Hardware

Cheaper HardwareHard drive capacity has fallen exponentially in price ever since Moore’s Law was introduced. This has changed attitudes to backups. The article says that today, hardware is so cheap and abundant that attitudes have shifted to a “Better keep this. We may need it someday” mentality.

New technologies, such as advancements in compression, deduplication and hardware virtualization, have improved overall storage utilization and further accelerated the rate at which the cost-per-gigabyte of storing data is falling.

Cheap and Abundant Bandwidth

Abundant BandwidthInternet bandwidth is no longer a bottleneck. Bandwidth availability has accelerated the growth of file sharing and online storage. Now large files are copied and distributed at an exponential rate which has caused duplicate data to become a major source of storage waste and data growth. The CEO of the firm based out of Toronto, calculates that if one person shares a 1GB file with 500 people, that’s half a terabyte of storage consumption.

Business is Going Paperless – Email has replaced letters, eBooks and tablets have nearly replaced paper books, and digital imaging has replaced photographs and x-rays. Not only are paperless offices better for the environment, but Mr. Rodin writes, they are also more productive, flexible and better able to extract value from their business data. Many industries are using more and more video (which is highly storage intensive) for marketing online, security and communication.

Enhanced Automated Data Collection Capabilities

Automated Data CollectionAutomated data collection is one of the fastest-growing areas in the “big data” space. With every move we make, the article says we’re generating GPS data, web traffic statistics, power usage data, surveillance video, and a broad range of data which companies and governments are collecting.

The author calls automated data collection the “Pandora’s Box” of the big data revolution. The information being collected about us through the electronic devices we use every day could present a threat to our privacy, but they also have the potential to offer tremendous value to society.

Advances in Data Analysis Technology

Data AnalysisThe blog says that until recently, data analysis was almost exclusively performed on structured relational databases, maintained and organized by humans. But now, a  new approach to data storage which focuses on rapid analysis and processing of vast data volumes. Technologies like Hadoop, Cassandra, MapReduce and NoSQL have given birth to a whole new class of services, and have revolutionized the way organizations think about the data they collect. Organizations can now get more insight into their internally generated business data by integrating external feeds and databases into their reporting and analysis.

The Growing Strategic Importance of Data

In the past, data was simply a tool which assisted in decision-making and helped companies execute on their strategic objectives. But recently Google (GOOG), Facebook (FB), Apple’s (AAPL) iTunes and other brands have built their entire corporate strategy around the data they own. The DCK article states, information is power, and it’s now more powerful than ever.

Regulatory ComplianceRegulatory Compliance

Even if companies wanted to cut the amount of data they store, they wouldn’t always be able to. Laws like PIPEDA, HIPAA, Sox404 and many others are forcing companies to keep historical archives of their exponentially growing business data going back several years.

As this data grows, storage increasingly becomes a major business problem. Also, companies must plan for cost-efficient search and retrieval of these large historical data volumes to stay ready for an unexpected electronic discovery request.

As the scale and complexity of big data storage grows, it’ll quickly reach a point where manual handling is no longer practical, desirable, economical, or even possible. Automation will become absolutely essential when it comes to backing up big data.

Many big data applications have serious privacy implications for the customers that benefit from their use. So security will become a top priority for backup administrators. Gone are the days of unencrypted backup tapes.

The big data applications has created a whole new class of applications built on real-time data. These applications require much more frequent  backups to optimize Recovery Point Objectives (RPOs). Strategic big data apps will need minimized downtime. This means smaller backup windows, built-in redundancy, and server fail-over to disaster recovery sites.

That’s why many organizations are opting to outsource their data backups by partnering with experts who run ahead of the trends and who can help with the complexity of some situations.

Related articles

 

Ralph Bach has been in IT long enough to know better and has blogged from his Bach Seat about IT, careers and anything else that catches his attention since 2005. You can follow him at LinkedIn, Facebook and Twitter. Email the Bach Seat here.

Category: Data protection | Tags: , , , , , , ,
| May 22, 2012
Comments Off on Holey Optochip Transfers TBPS Using Light

Holey Optochip Transfers TBPS Using Light

Holey Optochip Transfers TBPS Using LightIBM scientists will report on a prototype optical chipset, dubbed “Holey Optochip.” It is the first parallel optical transceiver to transfer one trillion bits – one terabit – of information per second. IBM will present the new chip at the 2012 Optical Fiber Communication Conference, conference for optical communications and networking professionals.

IBM old logoThe Holey Optochip is a standard silicon CMOS chip with holes punched in it. According to ITnewsLink, it was not possible to transport terabits of data for existing parallel optical communications technology until now. Reportedly the new IBM (IBM) chip prototype will deliver ultra-high interconnect bandwidth. The new compact chip will efficiently enable bandwidth to facilitate growth. Some of the segments it is targeted at are big data and cloud computing and next-generation data center applications.

Big Blue speculates that the chip could move information eight times faster than today’s systems. Speeds that fast could transform how data is accessed, shared, and used. It could open a new era of communications, computing, and entertainment. “Reaching the one trillion bit per second mark with the Holey Optochip marks IBM’s latest milestone to develop chip-scale transceivers that can handle the volume of traffic in the era of big data,” said IBM Researcher Clint Schow, part of the team that built the prototype.

Holey OptochipThe holes in Holey Optochip allow light through the chip. They produce an ultra-compact, high-performing, and power-efficient optical module. It is capable of record-setting data transfer rates.  ITnewsLink says optical networking can significantly improve data transfer rates by speeding the flow of data using light pulses, instead of sending electrons over wires. Researchers have searched for ways to make use of optical signals widespread within standard low-cost, high-volume chip manufacturing. The Holey Optochip module is constructed with commercially available components, providing the possibility to manufacture at economies of scale.

“We have been actively pursuing higher levels of integration, power efficiency, and performance for all the optical components through packaging and circuit innovations. We aim to improve on the technology for commercialization in the next decade with the collaboration of manufacturing partners,” Mr. Schow said in a press release.

Greem light bulbThe Holey Optochip is green

The Holey Optochip achieves its speed while consuming less than five watts. IBM claims the power consumed by a 100W light bulb could power 20 transceivers. This progress in power-efficient interconnects will allow companies who adopt high-performance computing to manage their energy load while performing powerful applications such as analytics, data modeling, and forecasting.

Technical Aspects of the Holey Optochip

The article explains that parallel optics is a fiber optic technology. It is primarily targeted at high-data, short-reach multimode fiber systems that are typically less than 150 meters. Parallel optics differ from traditional duplex fiber optic serial communication.  In parallel optics, data is simultaneously transmitted and received over multiple optical fibers.

Holey Optochip with optical viasA single 90-nanometer IBM CMOS transceiver IC becomes a Holey Optochip with the fabrication of forty-eight through-silicon holes, or “optical vias.” There is one optical via for each transmitter and receiver channel. Simple post-processing on completed CMOS wafers with all devices and standard wiring levels results in an entire wafer populated with Holey Optochips. The transceiver chip measures only 5.2 mm x 5.8 mm. Twenty-four channel, industry-standard 850-nm VCSEL (vertical cavity surface emitting laser) and photodiode arrays are directly flip-chip soldered to the Optochip. This direct packaging produces high-performance, chip-scale optical engines. The Holey Optochips are designed for direct coupling to a standard 48-channel multi-mode fiber array through an efficient microlens optical system that can be assembled with conventional high-volume packaging tools.

rb-

This one does not count as a new speed record – yet. It’s not real. Once Big Blue demonstrates Holey Optochip in the real world like this and this then it probably will be the fastest toy in town. The raw speed of one transceiver is equal to the bandwidth consumed by 100,000 users at today’s typical 10 Mb/s high-speed internet access.

At one terabit per second, IBM’s Holey Optochip will offer unprecedented amounts of bandwidth to move data like machine-to-machine communications (M2M) and other Internet of Things (IoT) components as well as posts to social media sites like Facebook (FB) and Twitter, videos to YouTube and digital pictures to Pinterest.

Mad scientistBut wait what if we use WDM within the light going thru Optochip.

Or better yet QAM 16 or even QAM 64

Or even more betterer QAM 256 running inside each wavelength of WDM.

 

Ralph Bach has been in IT long enough to know better and has blogged from his Bach Seat about IT, careers, and anything else that catches his attention since 2005. You can follow him on LinkedIn, Facebook, and Twitter. Email the Bach Seat here.

Category: Uncategorized | Tags: , , , , , , , , , ,
| November 8, 2011
One comment

Gartner’s Top Tech Trends For 2012

GartGartner's Top Tech Trends For 2012ner VP David Cearley described their top ten strategic technology trends for 2012 to attendees of the Gartner Symposium IT/Expo. Gartner (IT) defines a strategic technology as one with the potential for significant impact on the enterprise in the next three years. Here are Gartner’s top strategic technologies for 2012.

Gartner logoMedia tablets and beyond: Bring-your-own-technology (BYOT) at work has become the norm, not the exception.  By 2015 tablet shipments will reach around 50% of laptop shipments and Windows 8 will likely be in third place behind Google‘s (GOOG) Android and Apple’s (AAPL) iOS. The net result is that Microsoft‘s (MSFT) share of the client platform, be it PC, tablet, or smartphone, will likely be reduced to 60% and it could fall below 50%, Mr. Cearley says.

Apple iPadThe implication for IT is that the era of PC dominance with Windows as the single platform will be replaced with a post-PC era where Windows is one of a variety of environments IT will need to support. Gartner says it expects iOS/Android will dominate the market with 80% of tablets shipped by 2015. IT leaders need a managed diversity program to address multiple form factors, as well as employees bringing their own smartphones and tablet devices into the workplace.

Mobile-Centric Applications and Interfaces. User interfaces with windows, icons, menus and pointers which have been in place for more than 20 years are changing. The UI will be replaced by mobile-centric interfaces emphasizing touch, gesture, search, voice, and video. Applications themselves are likely to shift to more focused and simple apps that can be assembled into more complex solutions. By 2015 half the applications that would be written as native apps in 2011 will instead be delivered as Web apps.

The Internet of Things (IoT)Internet of Things: The Internet of Things (IoT) describes pervasive computing where cameras, sensors, microphones, image recognition, everything, is now part of the environment. In addition, increasingly intelligent devices create issues such as privacy concerns. Gartner says. Drivers of the IoT are:

  • Near Field Communication (NFC) payments allows users to make payments by waving their mobile phone in front of a compatible reader.
  • Embedded sensors which detect and communicate changes are being built into an increasing number of places and objects.
  • Image Recognition technologies identify objects, people, buildings, places logos, etc. that has value to consumers and enterprises.

App Stores and MarketplacesApp Stores and Marketplaces: Application stores by Apple and Android provide marketplaces where hundreds of thousands of applications are available to mobile users. Gartner forecasts that by 2014, there will be more than 70 billion mobile application downloads from app stores every year with an enterprise focus. With enterprise app stores, the role of IT shifts from that of a centralized planner to a market manager providing governance and brokerage services to users and potentially an ecosystem to support entrepreneurs. Enterprises should use a managed diversity approach to focus on app store efforts and segment apps by risk and value.

Big DataBig Data: The size, complexity of formats, and speed of delivery exceed the capabilities of traditional data management technologies; Gartner says it requires the use of new technologies simply to manage the volume alone. One major implication of big data is that in the future users will not be able to put all useful information into a single data warehouse. Logical data warehouses bringing together information from multiple sources as needed will replace the single data warehouse model.

Cloud Computing: This topic is still an important trend. It will become the next-generation battleground for the likes of Google and Amazon (AMZN). Going forward, enterprise IT will be concerned with developing hybrid private/public cloud apps, improving security and governance, Mr. Cearley says. While the market remains in its early stages in 2011 and 2012, it will see the full range of large enterprise providers fully engaged in delivering a range of offerings to build cloud environments and deliver cloud services. Oracle (ORCL), IBM (IBM), and SAP (SAP) all have major initiatives to deliver a broader range of cloud services over the next two years. As Microsoft continues to expand its cloud offering, and these traditional enterprise players expand offerings, users will see competition heat up and enterprise-level cloud services increase.

Cloud ComputingEnterprises are moving from trying to understand the cloud to making decisions on selected workloads to implement on cloud services and where they need to build out private clouds. Hybrid cloud computing which brings together external public cloud services and internal private cloud services, as well as the capabilities to secure, manage and govern the entire cloud spectrum will be a major focus for 2012. From a security perspective, new certification programs will be ready for the initial trial, setting the stage for more secure cloud computing. On the private cloud front, IT will be challenged to bring operations and development groups closer together using “DevOps” concepts in order to approach the speed and efficiencies of public cloud service providers.

Other key predictions Gartner had included:

  • Contextual and Social User Experience: Context-aware computing uses information about an end-user to improve the quality of interaction and anticipates the user’s needs and proactively serves up the customized content. By 2015, 40% of the world’s smartphone users will opt in to context service providers that track their activities with Google, Microsoft, Nokia (NOK), and Apple continuously tracking daily activities Mr.Cearley says.
  • The growing use of flash memory for In-Memory Computing is a long-term technology trend that could have a disruptive impact comparable to that of cloud computing.
  • The adoption of Extreme Low-Energy Servers built on low-power processors typically used in mobile devices will increase for non-compute intensive workloads or delivery of static objects to a website. Gartner says that 10%-15% of enterprise workloads are good for this.
  • Next-Generation Analytics Gartner says over the next three years, analytics will mature from structured and simple data analyzed by individuals to the analysis of complex information of many types (text, video, etc.) from many systems.
Related articles
  • Expecting a recession, Gartner urges ‘creative destruction’ (networkworld.com)

 

Ralph Bach has been in IT long enough to know better and has blogged from his Bach Seat about IT, careers, and anything else that catches his attention since 2005. You can follow him on LinkedIn, Facebook, and Twitter. Email the Bach Seat here.

Category: Uncategorized | Tags: , , , , , , , , , , , , , ,
  Recent Entries »