However, as systems grow more complex and the amount of data grows exponentially, so do the challenges in troubleshooting and maintaining them. Generative AI promises to improve the SRE experience and tame complexity. In particular, AI assistants based on retrieval augmented generation (RAG) are accelerating root cause analysis (RCA) and improving customer experiences.

The observability challenge

Observability provides complete visibility into system and application behavior, performance, and health using multiple signals such as logs, metrics, traces, and profiling. Yet, the reality often needs to catch up. DevOps teams and SREs frequently find themselves drowning in a sea of logs, metrics, traces, and profiling data, struggling to extract meaningful insights quickly enough to prevent or resolve issues. The first step is to leverage OpenTelemetry and its open standards to generate observability data in consistent and understandable formats. This is where the intersection of OpenTelemetry, GenAI, and observability becomes not just valuable, but essential.

RAG-based AI assistants: A paradigm shift 

RAG represents a significant leap forward in AI technology. While LLMs can provide valuable insights and recommendations leveraging public domain expertise from OpenTelemetry knowledge bases in the public domain, the resulting guidance can be generic and of limited use. By combining the power of large language models (LLMs) with the ability to retrieve and leverage specific, relevant internal information (such as GitHub issues, runbooks, customer issues, and more), RAG-based AI Assistants offer a level of contextual understanding and problem-solving capability that was previously unattainable. Additionally, the RAG-based AI Assistant can retrieve and analyze real-time telemetry from OTel and correlate logs, metrics, traces, and profiling data with recommendations and best practices from internal operational processes and the LLM’s knowledge base.

In analyzing incidents with OpenTelemetry, AI assistants that can help SREs:

1. Understand complex systems: AI assistants can comprehend the intricacies of distributed systems, microservices architectures, and the OpenTelemetry ecosystem, providing insights that take into account the full complexity of modern tech stacks.
2. Offer contextual troubleshooting: By analyzing patterns across logs, metrics, and traces, and correlating them with known issues and best practices, RAG-based AI assistants can offer troubleshooting advice that is highly relevant to the specific context of each unique environment.
3. Predict and prevent issues: Leveraging vast amounts of historical data and patterns, these AI assistants can help teams move from reactive to proactive observability, identifying potential issues before they escalate into critical problems.
4. Accelerate knowledge dissemination: In rapidly evolving fields like observability, keeping up with best practices and new techniques is challenging. RAG-based AI assistants can serve as always-up-to-date knowledge repositories, democratizing access to the latest insights and strategies.
5. Enhance collaboration: By providing a common knowledge base and interpretation layer, these AI assistants can improve collaboration between development, operations, and SRE teams, fostering a shared understanding of system behavior and performance.

Operational efficiency

For organizations looking to stay competitive, embracing RAG-based AI assistants for observability is not just an operational decision—it’s a strategic imperative. It helps overall operational efficiency through:

1. Reduced mean time to resolution (MTTR): By quickly identifying root causes and suggesting targeted solutions, these AI assistants can dramatically reduce the time it takes to resolve issues, minimize downtime, and improve overall system reliability.
2. Optimized resource allocation: Instead of having highly skilled engineers spend hours sifting through logs and metrics, RAG-based AI assistants can handle the initial analysis, allowing human experts to focus on more complex, high-value tasks.
3. Enhanced decision-making: With AI assistants providing data-driven insights and recommendations, teams can make more informed decisions about system architecture, capacity planning, and performance optimization.
4. Continuous learning and improvement: As these AI Assistants accumulate more data and feedback, their ability to provide accurate and relevant insights will continually improve, creating a virtuous cycle of enhanced observability and system performance.
5. Competitive advantage: Organizations that successfully leverage RAG AI Assistants in their observability practices will be able to innovate faster, maintain more reliable systems, and ultimately deliver better experiences to their customers.

Embracing the AI-augmented future in observability

The combination of RAG-based AI assistants and open source observability frameworks like OpenTelemetry represents a transformative opportunity for organizations of all sizes. Elastic, which is OpenTelemetry native, and offers a RAG-based AI assistant, is a perfect example of this combination. By embracing this technology, teams can transcend the limitations of traditionally siloed monitoring and troubleshooting approaches, moving towards a future of proactive, intelligent, and highly efficient system management.

As leaders in the tech industry, it’s imperative that we not only acknowledge this shift but actively prepare our organizations to leverage it. This means investing in the right tools and platforms, upskilling our teams, and fostering a culture that embraces AI as a collaborator in our quest to achieve the promise of observability.

The future of observability is here, and it’s powered by artificial intelligence. Those who recognize and act on this reality today will be best positioned to thrive in the complex digital ecosystems of tomorrow.

To learn more about Kubernetes and the cloud native ecosystem, join us at KubeCon + CloudNativeCon North America, in Salt Lake City, Utah, on November 12-15, 2024.

The post Accelerate root cause analysis with OpenTelemetry and AI assistants appeared first on SD Times.

If you can generate five or even 10 times the amount of code you previously could, that’s also five to 10  times more code that needs to be tested. 

“Many CFOs right now are looking at $30 per month per developer to go get them a GitHub Copilot or similar product,” said Jim Scheibmeir, senior director analyst at Gartner. “And I feel like we’ve kind of forgotten that frequently a bottleneck in software development is not the writing of code, but the testing of code. We’re gonna make developers so much more productive, which includes making them more productive at writing defects.”

Unlike AI-assisted dev tools where developers want to write more code, the goal with AI-assisted testing tools is to enable less testing. For instance, according to Scheibmeir, things like test impact analysis tools can create a testing strategy that is properly sized for the actual code change that is being pushed, so that only the tests that need to be run are run, rather than just running every test you have for every change. 

“These tools provide focus for testers,” he said. “And it’s so very difficult to give testers focus today. There’s this feeling like we must go test all of the things and yet we’re always crunched on time.”

Arthur Hicken, chief evangelist at Parasoft, agrees that we’ve already reached a point where test suites are taking hours, or even days, to complete, and using generative AI to help optimize test coverage can help with that.  “You can put together with AI these days a pretty good estimation of what you need to do to validate a change,” he said.

Generative AI helping with test generation, management, and more

Beyond helping testers test less, AI is creeping into other aspects of the process to make it more efficient end to end. For instance, Madhup Mishra, SVP at SmartBear, says that generative AI can now be used to create the tests themselves. “The tester can actually express their software test in simple English, and AI can actually create the automated test on their behalf,” he said. 

“Behind the scenes, GenAI should be understanding the context of the test, understanding what’s happening on the screen, and they can actually come up with a recommended test that actually solves the user’s problem without the user having to do a lot more,” he said.

Scheibmeir explained that the idea of making test generation easier had already been explored by low-code and no-code tools with their intuitive drag-and-drop interfaces, and generative AI is now taking it to that next level. 

And according to Eli Lopian, CEO of Typemock, AI is really good at exploring edge cases and may come up with scenarios that a developer might have missed. He believes that it can understand complex interactions in the codebase that the tester might not see, which can result in better coverage. 

AI can also help with generation of test data, such as usernames, addresses, PIN codes, phone numbers, etc. According to Mishra, generating test data can often be a lengthy, time-consuming process because testers have to think up all the possible variations, such as the characters that can go in a name or the country codes that come before phone numbers. 

“Generative AI can create all the different combinations of test data that you can ultimately use to be able to test all the corner cases,” Mishra explained. 

Another potential opportunity is using AI in test management. Companies often have a repository of all the different tests they have created, and AI can sort through all that and make suggestions on which to use. This allows testers to utilize what they’ve already created and free up more of their time to create new tests they need, explained Mishra. 

Parasoft’s Hicken added that AI could sort through older tests and validate if they are still going to work. For instance, if a test is capturing today’s date, then that test won’t work tomorrow. 

AI might make testing more accessible, but won’t eliminate need for it

Together, all of these AI enhancements are helping organizations take more responsibility for software quality themselves, where in the past they might have outsourced testing, Scheibmeir said. 

Similar to the citizen developer movement, the capabilities for testing that are now available make it easier for anyone to run a test, so it doesn’t require such specialized skills like it once did. 

“The hype and capabilities that generative AI are offering have brought some of these organizations back to the table of should we own more of that testing ourselves, more of that test automation ourselves,” Scheibmeir said. 

However, it’s still important to keep in mind that AI does have its drawbacks. According to Lopian, one of the biggest downsides is that AI doesn’t understand the emotion that software is supposed to give you. 

“AI is going to find it difficult to understand when you’re testing something and you want to see, is the button in the right place so that the flow is good? I don’t think that AI would be as good as humans in that kind of area,” he said.

It’s also important to remember that AI won’t replace testers, and testers will still need to keep an eye on it for now to ensure all the right coverage and the right tests are happening. Lopian likened it to a “clever intern” that you still need to keep an eye on to make sure they’re doing things correctly. 

AI’s impact on development skills will drive need for quality to shift further left

Another important consideration is the potential that if developers rely too heavily on generative AI, their development skills might atrophy, Mishra cautioned. 

“How many times have you gotten an Uber and realized the Uber driver knows nothing about where you’re going, they’re just blindly following the direction of the GPS, right? So that’s going to happen to development, and QA needs to sort of come up to speed on making sure that quality is embedded right from the design phase, all the way to how that application code will behave in production and observing it,” he said.  

Hicken agrees, likening it to how no one memorizes phone numbers anymore because our phones can store it all. 

“If I was a young person wanting to have a good long-term career, I would be careful not to lean on this crutch too much,” he said.

This isn’t to say that developers will totally forget how to do their jobs and that in 20, 30 years no one will know how to create software without the help of AI, but rather that there will emerge a new class of “casual developers,” which will be different from citizen developers.

Hicken believes this will lead to a more stratified developer community where you’ve got the “OG coders” who know how the computer works and how to talk to it, and also casual developers who know how to ask the computer questions — prompt engineers. 

“I think we are going to have to better define the people that are creating and managing our software, with roles and titles that help us understand what they’re capable of,” he said. “Because if you just say software engineer, that person needs to actually understand the computer. And if you say developer, it might be that they don’t need to understand the computer.”

You may also like…

The evolution and future of AI-driven testing: Ensuring quality and addressing bias

RAG is the next exciting advancement for LLMs

The post Generative AI development requires a different approach to testing appeared first on SD Times.

Models are tested using a metric called context adherence, which measures “closed-domain hallucinations: cases where your model said things that were not provided in the context.”

The best performing model overall for RAG, according to the ranking, is Claude 3.5 Sonnet from Anthropic. Galileo said that this model and Anthropic’s other model Claude 3 Opus had near perfect scores, beating out OpenAI’s models, which won last year. 

From a cost perspective, the best performing model was Google’s Gemini 1.5 Flash. And Alibaba’s Qwen2-72B-Instruct was overall the best performing open source model, though in short context RAG tests, Meta’s llama-3-60b-instruct was the best. 

Broken down by context length, the best closed-source model in short context RAG was Claude 3.5 Sonnet, in medium context RAG was Google’s Gemini-1.5-flash-001 (with cost being the tiebreaker with other models that also scored a perfect score), and in large context RAG was again Claude 3.5 Sonnet. 

“In today’s rapidly evolving AI landscape, developers and enterprises face a critical challenge: how to harness the power of generative AI while balancing cost, accuracy, and reliability. Current benchmarks are often based on academic use-cases, rather than real-world applications. Our new Index seeks to address this by testing models in real-world use cases that require the LLMs to retrieve data, a common practice in enterprise AI implementations,” says Vikram Chatterji, CEO and co-founder of Galileo. “As hallucinations continue to be a major hurdle, our goal wasn’t to just rank models, but rather give AI teams and leaders the real-world data they need to adopt the right model, for the right task, at the right price.”

You may also like…

Anthropic’s new Claude 3.5 Sonnet model already competitive with GPT-4o and Gemini 1.5 Pro on multiple benchmarks

Meta’s new Llama 3.1 model competes with GPT-4o and Claude 3.5 Sonnet

The post Claude 3.5 Sonnet comes out on top in Galileo’s Hallucination Index appeared first on SD Times.

The initiative was spearheaded by integration platform Boomi, professional services firm Connor Group, and a number of professors, including David Wood from Brigham Young University, Scott Emett from Arizona State University, and Marc Eulerich from University of Duisberg-Essen.

According to the creators, the framework can be utilized by companies of any size and can be modified to suit specific objectives, needs, and risk appetites. 

“Effective AI adoption will be a massive competitive advantage, but many don’t know where to start and how to apply it,” said Jeff Pickett, chair and founder at Connor Group. “A few AI tools exist now, with many on the way, and they are coming fast. Having a smart AI adoption strategy with underlying controls, data, and processes that are ready for AI takes time. The most competitive companies are doing these things now.”

The Enterprise GenAI Governance Framework also works well alongside the GenAI Governance Framework Maturity Model (also created jointly by Wood, Emett, and Eulerich), which allows businesses to evaluate their current governance practices, find areas for improvement, and plan for those improvements to be implemented. 

“Overall, the GenAI Governance Framework and accompanying Maturity Model serve as essential resources for organizations seeking to navigate the evolving landscape of AI,” said Wood. “By adopting these tools, organizations can enhance their preparedness, resilience, and capability to harness the benefits of AI while effectively managing its risks.”

You may also like…

Report: Ethical AI use will be a competitive business advantage

Discerning reality from the hype around AI

Dataiku helping companies prepare for EU AI Act with new readiness program

The post 1,000+ business leaders collaborate to create Enterprise GenAI Governance Framework appeared first on SD Times.

The six-week cohort learning program, Unlocking GenAI Opportunities with AWS, will teach leaders strategies on how to make the most of generative AI in their organizations. It is designed for VP and director-level employees. 

According to Udemy, business leaders need to master these three skills in order to effectively leverage generative AI: proficiency with generative AI technology, change management skills to create an innovation culture, and effective communication and collaboration.  

Throughout the six weeks, leaders taking the course will learn topics related to these skills. In the area of generative AI, they will learn how it can improve productivity, ethical uses, and prompt engineering techniques, as well as take part in a hands-on lab where they will transform a fictitious company using the technology. 

They will also learn skills related to being able to locate innovation opportunities and how to execute them. And finally, there will be two weeks dedicated to how to communicate proposed changes effectively so that initiatives can be successfully implemented. 

“Generative AI is the most transformative technology we’re likely to see in our lifetimes. Leaders who build their knowledge, skills, and a mindset of resilience and adaptability, will position themselves—and their organizations—to harness its immense potential,” said Maureen Lonergan, vice president of training and certification at AWS. “We’re excited to work with Udemy to realize a shared vision of building a global, AI-ready workforce. Unlocking GenAI Opportunities with AWS is an ideal starting point for leaders everywhere to build their generative AI strategy and set their organization on a path of accelerated innovation.”

Greg Brown, president and CEO of Udemy, added: “We are thrilled to be working in collaboration with AWS to support organizations around the world, helping to drive strategic outcomes and inspire their teams to achieve new heights.”

The post Udemy and AWS launch course to teach business leaders about generative AI appeared first on SD Times.

One of the goals of the study is to use the results to assist people in making these determinations in their day-to-day lives. 

The results of the study could also be used to create content authenticity detection technologies and technologies that can identify the source of fake information. 

NIST GenAI will involve Generator teams and Discriminator teams. The Generator teams will be evaluated on their system’s ability to use AI to generate content that can’t be differentiated from human-generated content. The Discriminator teams will be evaluated on their system’s ability to detect AI-generated content. 

NIST’s GenAI is part of the response to President Biden’s Executive Order on AI and the results will inform the work of NIST’s U.S. AI Safety Institute. 

“Pilot evaluations provide valuable lessons for future research on cutting-edge technologies and guidance for responsible and safe use of digital content,” the NIST GenAI website states. 

The post NIST GenAI program launches to study how to distinguish AI-generated and human-generated content appeared first on SD Times.

According to Atlassian, the basis of Atlassian Rovo is the teamwork graph, which is a data model the company created based on the understanding it has gained over the past couple of decades on common work patterns, anti-patterns, organizational structures, and lines of communication. The teamwork graph pulls in data from Atlassian apps and other SaaS apps connected to it.

“AI is only as useful as the data it taps into … With every new tool connection, team action, and project event, teamwork graph draws more connections and expands its knowledge to deliver increasingly relevant results,” Jamil Valliani, head of AI product at Atlassian, wrote in a blog post. 

Rovo Search returns contextual, relevant results based on information from different tools, including Google Drive, Microsoft Sharepoint, Microsoft Teams, GitHub, Slack, Figma, or even tools that were developed internally. 

Information is presented in knowledge cards, which provide a quick glance of information on projects, goals, team members, and more. 

“Teams get immediate answers as they work, and knowledge cards get smarter as more data is added to the Atlassian teamwork graph,” Valliani wrote. 

Rovo also has Agents, which can actually execute specific tasks, such as taking action when as Jira issue progresses, create service checklists, assist with new employee onboarding, organize Confluence pages, and more. 

“Rovo Agents will transform teamwork with their ability to synthesize large volumes of enterprise data, break down complex tasks, learn as they take action, and partner with their human teammates to make critical and complex decisions. Agents aren’t just some souped-up version of chatbots. They bring specialized knowledge and skills to a wide variety of workflows and processes,” said Valliani. 

Currently there is a waitlist to get access to Atlassian Rovo. 

The post Atlassian Rovo makes it easier to understand, take action on enterprise data across different SaaS tools appeared first on SD Times.

The developer version can assist with tasks such as code generation, testing, debugging, while the business version can be used to answer questions from company data, such as policies, product information, business results, or employee data.

According to AWS, only 30% of a developer’s time is spent actually coding, with most of the time spent doing things like learning from documentation or forums, managing infrastructure or resources, troubleshooting errors, refactoring code, and more.  

“Since we announced the service at re:Invent, we have been amazed at the productivity gains developers and business users have seen. Early indications signal Amazon Q could help our customers’ employees become more than 80% more productive at their jobs; and with the new features we’re planning on introducing in the future, we think this will only continue to grow,”  Dr. Swami Sivasubramanian, vice president of Artificial Intelligence and Data at AWS.

Amazon Q Developer can utilize the company’s own codebase to gain a deeper understanding of developer requests in the context of what they are already building to provide more personalized results.

AWS claims that based on customers who have already been using Amazon Q Developer, it has the highest acceptance rate for generated code of any AI assistant. The company says that BT Group accepted 37% of the suggestions and National Australia Bank accepted 50% of suggestions.

In addition to the assistant part of Amazon Q Developer, it also features “agents,” which can autonomously perform tasks like feature implementation, documentation, code refactoring, or performing software upgrades. For example, a developer could ask Q to create an “add to favorites” button in a social sharing app and Q will generate a step-by-step implementation plan. The developer then reviews the plan and can alter it before giving Q the approval to move forward with implementing its plan.

It also offers security scanning capabilities to detect vulnerabilities present in the codebase, like exposed credentials or log injection. 

Additionally, being an AWS product, it has a deep understanding of a company’s AWS environment and can make improvements to help optimize it. For instance, it can diagnose and resolve networking errors, optimize SQL queries and ETL pipelines, and provide architectural best practices. It also provides easy visibility into account resources, configurations, and billing information and trends. 

AWS says this allows it to answer queries such as “What instances are currently running in US East 1?” or “What’s my S3 bucket encryption?” or “What were my EC2 costs by region last month?” 

“Amazon Q is the most capable generative AI-powered assistant available today with industry-leading accuracy, advanced agents capabilities, and best-in-class security that helps developers become more productive and helps business users to accelerate decision making,” said Sivasubramanian.

The post AWS’ generative AI assistant Amazon Q now generally available appeared first on SD Times.

The Generative AI Copyright Disclosure Act would make it so that companies need to submit a notice to the Register of Copyrights before they release an AI system that was trained on copyrighted works. This would also retroactively apply to models previously released that fit the requirements.

“AI has the disruptive potential of changing our economy, our political system, and our day-to-day lives. We must balance the immense potential of AI with the crucial need for ethical guidelines and protections. My Generative AI Copyright Disclosure Act is a pivotal step in this direction. It champions innovation while safeguarding the rights and contributions of creators, ensuring they are aware when their work contributes to AI training datasets. This is about respecting creativity in the age of AI and marrying technological progress with fairness,” said Schiff.

It has received support from a number of organizations — particularly in the creative space — including the Directors Guild of America, SAG-AFTRA, both East and West chapters of the Writers Guild of America, and the American Society of Composers, Authors and Publishers (ASCAP), to name a few.

“Everything generated by AI ultimately originates from a human creative source. That’s why human creative content—intellectual property—must be protected. SAG-AFTRA fully supports the Generative AI Copyright Disclosure Act, as this legislation is an important step in ensuring technology serves people and not the other way around,” said Duncan Crabtree-Ireland, national executive director and chief negotiator at SAG-AFTRA.

Elizabeth Matthews, CEO of ASCAP, added: “Without transparency around the use of copyrighted works in training artificial intelligence, creators will never be fairly compensated and AI tech companies will continue stealing from songwriters. This bill is an important step toward ensuring that the law puts humans first, and we thank Congressman Schiff for his leadership.”

The post New legislation would require companies to disclose that they are training AI models on copyrighted work appeared first on SD Times.

“With Copilot, we have united everything AI in one place,” said Andie Dovgan, chief growth officer at Creatio.” We have the ability to configure all sorts of different AI use cases, and roll them out to applications and all sorts of different things that you do with the Creatio platform.”

RELATED CONTENT: How GenAI and Low-Code are Turbo-Charging Software Development

Creatio Copilot comes with Copilot Studio, which is an area that includes a number of pre-built automations, such as a knowledge base assistant, appointment scheduling, and a contract renewal reminder. The pre-built automations can be further customized to better suit a customer’s specific needs, or Copilot Studio can also be used to can create new automations entirely. 

“So within our concept of Gen AI integration, we fully leverage the idea of no code,” Dovgan explained. “We allow the end user to configure use cases for Gen AI similarly to how we enable them to configure anything else in our platform. So it’s not just that we’ve developed a lot of different use cases and we kind of sprinkle them around our application as a platform; we created a toolkit to provides the end user with the ability to get into that scenario, change the prompt and additional action items, get into the advanced settings and really tweak and adjust to meet their specific requirements.”

The pre-built automations are mainly designed for development, sales professionals, marketers, and customer service agents in mind, though anyone can create their own automation to suit different needs. 

Developers can use the generative AI capabilities to describe the type of application they want to create and then have it generate the UX components. Then, they can go in and add the specific capabilities they want for their application. 

For sales, Creatio Copilot can be used to do things like identify and score engaged customers, generate conversation summaries, and get prompts for the next action to take.

For marketers, it can help design optimized campaign flows, select the most responsive audiences for campaigns, and prepare communications templates. 

And for customer service agents, Creatio Copilot can analyze customer emails, determine the urgency and category of cases, and suggest the most qualified agent to assign a ticket to. 

“We’re combining no-code and Gen AI, and we give an incredible set of opportunities for non-technical people to build those use cases on the fly,” said Dovgan. “And then we’ll also provide a rich set of out of the box and Gen AI use cases through Copilot for sales, marketing, customer service and assisted development.”

The post Creatio Copilot brings generative AI to no-code workflow automation appeared first on SD Times.