This article is contributed. See the original author and article here.
The most accurate way to model your use case is to simulate the load you expect on your own hardware. You can do this using the load generation tools that ship with Kafka
However, if you want to size a cluster without simulation, a very simple rule could be to size the cluster based on the amount of disk-space required (which can be computed from the estimated rate at which you get data times the required data retention period).
A slightly more sophisticated estimation can be done based on network and disk throughput requirements. To make this estimation, let’s plan for a use case with the following characteristics:
W– MB/sec of data that will be written
R– Replication factor
C– Number of consumer groups, that is the number of readers for each write
Kafka is mostly limited by the disk and network throughput.
The volume of writing expected isW * R(that is, each replica writes each message). Data is read by replicas as part of the internal cluster replication and also by consumers. Because every replicas but the master read each write, the read volume of replication is(R-1) * W. In addition each of theCconsumers reads each write, so there will be a read volume ofC * W. This gives the following:
Writes:W * R
Reads:(R+C- 1) * W
An easy way to model the real consumer lagging is to assume a number of lagging readers you to budget for. To model this, let’s call the number of lagging readersL. A very pessimistic assumption would be thatL = R + C -1, that is that all consumers are lagging all the time. A more realistic assumption might be to assume no more than two consumers are lagging at any given time.
Based on this, we can calculate our cluster-wide I/O requirements:
Disk Throughput (Read + Write):W * R + L * W
Network Read Throughput:(R + C -1) * W
Network Write Throughput:W * R
Deciding Number of Disks
In HDInsight Kafka cluster you can choose the number of disks (only) during the cluster creation. The number varies based on VM SKU, the size of each disk is 1 TB.
eg : For E8aV4, once can attach up to 16 disks where as for D12V2 its 8.
Deciding Number of Brokers/Workers
If the selected VM have a 1 Gigabit Ethernet card with full duplex, then that would give 125 MB/sec read and 125 MB/sec write. Likewise you can calculate the total read-write capacity per machine. Once we know the total requirements, as well as what is provided by one machine, you can divide to get the total number of machines needed.
Note : This gives a machine count running at maximum capacity, assuming no overhead for network protocols, as well as perfect balance of data and load. Since there is protocol overhead as well as imbalance, you want to have at least 2x this ideal capacity to ensure sufficient capacity.
This article is contributed. See the original author and article here.
Delays, constraints, and disruptions are frequent challenges for global supply chains. And they are costly. According to Gartner 70 percent of supply chain leaders report that they have been constantly responding to disruptions since 2019.1 Supply chain professionals have long understood that the earlier potential disruptions and constraints are identified, the more successful the measures to adapt and overcome them can be. So, the question becomes, how do we identify disruptions and constraints at the earliest possible opportunity? The answer is real-time, end-to-end visibility across the supply chain, which is precisely what supply chain control towers provide.
While the benefits of supply chain control towers are easily understood, many organizations continue to struggle to make control towers a reality. The composable approach of Microsoft Dynamics 365 is changing this situation however, and providing organizations a path to rapid deployment of supply chain control towers. This article explains the composable approach and why it is an essential ingredient for success in supply chain control tower initiatives.
The future of business applications is composable. We can see this in reports from analyst firms such as Gartner, who predict that “by 2023 organizations, that have adopted a composable approach will outpace their competition by 80 percent.”2 But, what is a composable approach? Before we delve into the composable approach and what makes it different, let’s briefly explore the evolution of enterprise resource planning (ERP) solutions as this will help us understand why composability is key to achieving end-to-end visibility.
The challenge of achieving end-to-end visibility
In the past, ERP systems were sold as one massive application. This meant that they were expensive and difficult to change. As these applications matured, vendors looked for ways to give customers greater flexibility. To provide this flexibility, business functions, like sales and supply chain, were increasingly offered as modular components such as customer relationships management (CRM) systems, warehouse management systems (WMS), transportation management systems (TMS), and supply chain management (SCM) systems. While modularity did provide flexibility, the flexibility itself created other problems, as we discuss next.
The main issue is that once business functions could select the components that best suited their needs, such as CRM or WMS, replacement and implementation of these systems often took place at different times, and systems were often from different vendors. Also, due to the possible differences in maturity levels and solution architectures of these disparate systems, they required time-consuming and expensive custom development to achieve end-to-end integration across the enterprise. And after integration, end-to-end performance was often plagued by latency, synchronization, and data unification issues. This is the situation where many companies find themselves today and it is the primary reason that achieving real-time end-to-end visibility of the supply chain has been so elusive.
Microservices are composable
Technology vendors began offering business solutions assembled application experiences built on microservices architectures in the continuing drive to improve customer applications. Whereas in the past, a specific business function, like warehouse or sales, would select the right system for their needs, such as CRM or WMS, businesses can now choose solutions tailored to individual workloads. For example, a simple order management workload could be composed by bringing together order orchestration, fulfillment, and inventory.
These microservices, such as real-time inventory and fulfillment, not only provide greater flexibility, but because they easily connect to other microservice applications and utilize API-driven messaging, data and actions can be pulled and pushed between different systemsdecreasing latency, improving synchronization, and enabling real-time end-to-end visibility in the process. This is why a composable approach makes the rapid deployment of a supply chain control tower possible.
Our vision
Gartner predicts that by 2026, 80 percent of companies will suffer significant value loss due to failure to merge their digital supply twin and control tower initiatives.3 We agree. And, our solutions help companies avoid this outcome by providing a composable approach that makes merging these initiatives possible.
Dynamics 365 Intelligent Order Management is often a great place to start when building a supply chain control tower in a retail, distribution, or direct-to-consumer (DTC) manufacturing business. This is because Dynamics 365 Intelligent Order Management is built on a microservice architecture that includes rules-based order orchestration and fulfillment, real-time inventory, and AI. As such, it is step-forward to the composable approach that allows companies to achieve the real-time end-to-end visibility of a supply chain control tower. The composability of the solution also means that organizations can get up running quickly and maximize the benefit of existing investments through easy integration with existing legacy ERP systems.
What’s next
Supply chain control towers that include Dynamics 365 Intelligent Order Management positions organizations to respond faster and more intelligently to disruptions and opportunities. Dynamics 365 Intelligent Order Management works seamlessly with any ERP and CRM system. It is also a stand-alone application that allows customers to start their control tower initiative without dependencies on other Dynamics 365 solutions. It integrates seamlessly with many market-leading API-enabled applications, using our out-of-the-box pre-built connectors to order intake sources, tax calculation, and third-party logistics partners ecosystem. Moreover, it automates and optimizes fulfillment using AI and real-time omnichannel inventory data. Supply chain and commerce professionals can proactively overcome disruptions through quickly order flows reconfigurations with drag and drop actions in a low-code/no-code interface. And by enabling them with all these capabilities, help them accelerate their order management digital transformation and turn order fulfillment into a competitive advantage.
GARTNER is the registered trademark and service mark of Gartner Inc., and/or its affiliates in the U.S. and internationally and has been used herein with permission. All rights reserved.
This article was originally posted by the FTC. See the original article here.
It’s almost February and you know what that means…it’s getting close to National Consumer Protection Week (NCPW)! This year, NCPW is March 6-12, 2022, so now’s the time to jump into planning.
NCPW is the time of year when government agencies, consumer protection groups, and people like you work together to help remind your family, friends, and neighbors of their consumer rights and avoid frauds and scams. Want to join in? Here are some ideas:
Help your family, friends and community avoid scams.Order free materials to share in English or Spanish. Order by February 7th to ensure delivery by NCPW.
Plan a virtual consumer protection event. Find ideas at ftc.gov/ncpw for how to get involved.
Share resources on COVID-19 scams. Share ways to avoid these scams, from COVID tests to vaccine scams. Check out ftc.gov/coronavirus/scams for free resources, including one-page handouts and graphics to share on social media.
This article was originally posted by the FTC. See the original article here.
Today we’re kicking off Identity Theft Awareness Week 2022. Identity thieves have been busier than ever during the pandemic, with scammers and identity thieves after people’s information to apply for credit, unemployment benefits, file taxes, buy things, or get medical services. But there are some things you can do to protect yourself, and this week, you’ll learn how.
Each day this week, we’ll cover topics from the steps you can take to protect your personal information, to what to know about credit freezes and fraud alerts. Follow along with these alerts to help keep your money and personal information safe from scammers.
But wait, there’s more! This week, tune into free webinars, podcasts, and other events to hear experts discussing how to spot, protect against, and recover from identity theft. There’s something for everyone — including some events that spotlight resources for active duty service members, veterans, and older adults.
Start by joining a Facebook Live discussion at 3 p.m. ET today: Impersonator Scams & Identity Theft. You’ll hear from experts from the FTC and AARP’s Fraud Watch Network about impersonators who pretend to be from the government or well-known businesses, how they try to steal your information, and how to spot and stop them.
For details on the week’s webinars, podcasts, and Facebook Live programs, as well as how to participate, visit Identity Theft Awareness Week 2022. Then join us on Friday, when we’ll wrap up with a Twitter chat discussing identity theft trends, advice on spotting and avoiding identity theft, and how to recover.
Learn more about identity theft at ftc.gov/idtheft. And, if identity theft happens to you or someone you know, visit IdentityTheft.gov to report it and get a personalized recovery plan.
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.
This article is contributed. See the original author and article here.
Final Update: Sunday, 30 January 2022 15:00 UTC
We’ve confirmed that all systems are back to normal with no customer impact as of 01/30, 14:25 UTC. Our logs show the incident started on 01/30, 13:25 UTC and that during the 1 hour that it took to resolve the issue some of customers using Azure Log Analytics or Azure Sentinel might have experienced delayed or missed Log Search Alerts and issues accessing data in West US2 region.
Root Cause: The failure was due one of our backend service was unhealthy.
Incident Timeline: 1 Hour – 01/30, 13:25 UTC through 01/30, 14:25 UTC
We understand that customers rely on Azure Log Analytics or Azure Sentinel as a critical service and apologize for any impact this incident caused.
Today we’re kicking off Identity Theft Awareness Week 2022. Identity thieves have been busier than ever during the pandemic, with scammers and identity thieves after people’s information to apply for credit, unemployment benefits, file taxes, buy things, or get medical services. But there are some things you can do to protect yourself, and this week, you’ll learn how.
Recent Comments