Categories
discuss

Kafka Dependencies – ccs vs ce

To develop my Kafka connector I need to add a connect-API dependency.

Which one I should use?

For example mongodb connector use connect-api from maven central

But links from dev guide go to https://packages.confluent.io/maven/org/apache/kafka/connect-api/5.5.0-ccs/ and beside 5.5.0-ccs there is also 5.5.0-ce version.

So, at this moment last versions are:

What difference between all three variants?

Which one I should use?

Answer

The 5.x version refer to Releases from Confluent whereas the 2.5.0 refers to the Open Source Apache Kafka project.

According to this documentation the ce belongs to the complete Confluent Platform, including all community and commercial component packages and the ccs to the Confluent Community component packages.

This doc on licenses around Confluent/Kafka will give you more details.

According to Confluent documentation on inter-compatibility:

Confluent Platform and Apache Kafka Compatibility

Confluent Platform  Apache Kafka
6.1.x               2.7.x
6.0.x               2.6.x
5.5.x               2.5.x
5.4.x               2.4.x
5.3.x               2.3.x
...                 ...

Kafka-Connect is part of Open Source Apache Kafka and of both Confluent packages, so it would not matter which version to use. There is a difference when it comes to pre-build/supported connectors available in the confluent packages compared to Open source version. As you plan to build your own, it would not matter.

This blog post is a nice introduction to Confluent Hub.

Categories
discuss

Can I use `expect.stringContaining()` inside a Jest `.toHaveBeenCalledWith()` block?

Is it possible to use expect.stringContaining() inside a Jest .toHaveBeenCalledWith() block?

I am currently using:

expect(createChatRoomMock).toHaveBeenCalledWith({
  creatorId: expect.stringContaining("user_"),
  chatRoomUuid: expect.stringContaining("chatRoom_"),
});

But this fails with:

    - Expected
    + Received


    Object {
  -   "chatRoomUuid": StringContaining "chatRoom_",
  -   "creatorId": StringContaining "user_",
  +   "chatRoomUuid": "chatRoom_sZ9nj4hC46e4bGz4PjYzpC",
  +   "creatorId": "user_nCQsasvYirUwwoEr3j8HsC",
    },

This is odd, as you can see from the error, the recieved strings match what’s expected

I’ve also tried:

expect(createChatRoomMock).toHaveBeenCalledWith({
  creatorId: expect.stringMatching(/user_.*/),
  chatRoomUuid: expect.stringMatching(/chatRoom_.*/),
});

With the same results as shown above.

How can I use expect.stringContaining() inside a Jest .toHaveBeenCalledWith() block?

Answer

This is a bug in jest. If there is anything else failing in the test, Jest will show these as failures, even though they would pass, for example:

  it.only("Test", () => {
    var createChatRoomMock = jest.fn();

    createChatRoomMock({
        "chatRoomUuid": "chatRoom_sZ9nj4hC46e4bGz4PjYzpC",
        "creatorId": "user_nCQsasvYirUwwoEr3j8HsC",
        "somethingElse": "bad"
    });

    expect(createChatRoomMock).toHaveBeenCalledWith({
      creatorId: expect.stringContaining("user_"),
      chatRoomUuid: expect.stringContaining("chatRoom_"),
      somethingElse: expect.stringContaining("good")
    });
  });

Will (inaccurately) show that the other .toHaveBeenCalledWith() have failed:

    - Expected
    + Received

      Object {
    -   "chatRoomUuid": StringContaining "chatRoom_",
    -   "creatorId": StringContaining "user_",
    -   "somethingElse": StringContaining "good",
    +   "chatRoomUuid": "chatRoom_sZ9nj4hC46e4bGz4PjYzpC",
    +   "creatorId": "user_nCQsasvYirUwwoEr3j8HsC",
    +   "somethingElse": "bad",
      },
Categories
discuss

Define a function inside useEffect or outside?

Why the fetchData function is defined inside the useEffect and not outside ?

Link: https://github.com/zeit/next.js/blob/canary/examples/with-graphql-faunadb/lib/useFetch.js

import { useState, useEffect } from 'react'

export default function useFetch(url, options) {
  const [data, setData] = useState(null)
  const [error, setError] = useState(null)

  useEffect(() => {
    const fetchData = async () => {
      try {
        const res = await fetch(url, options)
        const json = await res.json()

        setData(json)
      } catch (error) {
        setError(error)
      }
    }
    fetchData()
    // eslint-disable-next-line react-hooks/exhaustive-deps
  }, [url])

  return { data, error }
}

I would have done that:

import { useState, useEffect } from 'react'

export default function useFetch(url, options) {
  const [data, setData] = useState(null)
  const [error, setError] = useState(null)

  // Defined outside of useEffect
  // `u` instead of `url` for not overlapping
  // with the one passed in useFetch()
  const fetchData = async (u) => {
    try {
      const res = await fetch(u, options)
      const json = await res.json()

      setData(json)
    } catch (error) 
      setError(error)
    }
  }

  useEffect(() => {
    // Using url as an argument
    fetchData(url)
    // eslint-disable-next-line react-hooks/exhaustive-deps
  }, [url])

  return { data, error }
}

It seems easier to read and better organized. I’m thinking it’s maybe an anti-pattern or something else ?

Answer

I typically define the functions inside the useEffect, there are several reasons for that

  1. By definining the function outside of the use effect, you either need to disable exhaustive-deps and risk accidentally having a stale function or you need to useCallback to make the function not-update every render
  2. If the function is only used in the useEffect, you don’t need to recreate the function on every render as that’s just wasted cycles
  3. It’s easier to work with cleanup on the asynchronous functions by defining it within useEffect as you can define variables that are able to be modified within the effect.

On that last one, for instance, you can do some actions to prevent state being called when the effect cleans up.

You could also use AbortController with fetch to cancel the fetch.

import { useState, useEffect } from 'react'

export default function useFetch(url, options) {
  const [data, setData] = useState(null)
  const [error, setError] = useState(null)

  useEffect(() => {
    let isUnmounted = false;
    const fetchData = async () => {
      try {
        const res = await fetch(url, options)
        const json = await res.json()
        if(!isUnmounted) setData(json)
      } catch (error) {
        if(!isUnmounted) setError(error)
      }
    }
    fetchData()
    return ()=>{isUnmounted = true;}
    // eslint-disable-next-line react-hooks/exhaustive-deps
  }, [url])

  return { data, error }
}
Categories
discuss

How can I simplify `(variableA && !variableB) || !variableA` expression in JavaScript?

Please see this minimum example:

const result = (variableA && !variableB) || !variableA;

In this expression, I can’t simply write this

const result = variableA && !variableB;

Because if variableA = 0, the result will be different

const variableA = 0;
const variableB = undefined;

console.log((variableA && !variableB) || !variableA); // true
console.log(variableA && !variableB); // 0

Is there any way I can simplify this expression?

Answer

You could use

!(a && b)

or the equivalent with De Morgan’s laws

!a || !b

const
    f = (a, b) => (a && !b) || !a,
    g = (a, b) => !(a && b),
    h = (a, b) => !a || !b

console.log(0, 0, f(0, 0), g(0, 0), h(0, 0));
console.log(0, 1, f(0, 1), g(0, 1), h(0, 1));
console.log(1, 0, f(1, 0), g(1, 0), h(1, 0));
console.log(1, 1, f(1, 1), g(1, 1), h(1, 1));
Categories
discuss

Unexpected Kafka request of type METADATA during SASL handshake

I am trying to connect Kafka Java Client to a Kafka broker using SASL Plain. But when I try to send a message from the Producer, the Kafka Server logs the following error:

[2020-04-30 14:48:14,955] INFO [SocketServer brokerId=0] Failed authentication with /127.0.0.1 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)

By the looks, the producer tries to send a metadata request, before the SASL handshake. How can I do the handshake before sending the message?

Following is my kafka_server_jaas.conf file, which is used for Kafka Server.

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret"
    user_admin="admin-secret";
};

Client {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret";
};

Following is my zookeeper_jaas.conf file, which is used for the zookeeper:

Server {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret"
    user_admin="admin-secret";
};

In my Java producer, I set the following properties:

Properties properties = new Properties();
properties.put("bootstrap.servers", "localhost:9092");
properties.put("sasl.jaas.config", "org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin_secret"");
properties.put("sasl.mechanisms", "PLAIN");
properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer kafkaProducer = new KafkaProducer(properties);

Is there anything wrong I am doing?

Answer

You need to specify security.protocol, otherwise by default, Kafka clients do not use SASL.

In your client properties, add:

properties.put("security.protocol", "SASL_SSL");

There’s also SASL_PLAINTEXT but it’s not recommended to use the PLAIN mechanism over SASL_PLAINTEXT as effectively the password will be exchanged in cleartext.

Source: stackoverflow
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Privacy Policy, and Copyright Policy. Content is available under CC BY-SA 3.0 unless otherwise noted. The answers/resolutions are collected from stackoverflow, are licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0 © No Copyrights, All Questions are retrived from public domain..