I didn't want to associate this account w/ my real name but now that you mentioned it wasn't right of me to not point that out. I added a disclaimer.
The posted algorithm and the one mentioned in my paper are very similar. It is just that the cumulative sum computation is parallelized in the posted website.
I think there's an issue with the histogram rendering in this post. The rapid descent from the spike on the left is not consistent with high ECDF impact and the apparent binning resolution visible in the piecewise line-segments. In general histograms should not be visualized with connected line-graphs in this way - the standard bar graph depiction makes the bin-width apparent and resolves some of the issues the article needs the ECDF for (e.g. relative impact can be assessed visually by comparing the relative areas of the associated bars). The bar visualization also makes it possible to use varying bin sizes, which is extremely useful with any distribution that has tails.
FYI some of the Airflow issues are out of date / can be resolved with config changes.
AirFlow 2 is designed to support larger XCOM messages, so the guidance to only use it for small data no longer applies.
Your DAG construction overhead issue is likely due to dagbag refreshing. Airflow checks for DAG changes on a fixed interval, causing a reimport. The default period for that is fairly small, so for large deployments you will want to use a larger period (e.g. at least 5 minutes). I do not know why the default is so short (or was last I checked, anyway). Python files shouldn't do much of note on import regardless IMO.
I am not otherwise familiar with the improvements in Airflow 2, so I cannot say for sure if your other complaints still remain.
I know that that some issues are fixed in Airflow 2, they have made a large improvement with that release. But not all issues is resolved with this one.
The performance issue is still here, just launch Airflow and submit thousand dagruns with simple python sleep(1) and you will hit the cpu bound very quickly with a total time that will have a large duration. Airflow is not designed for a lot of short duration tasks. When using event driving data flow, it's really complicated to managed.
Imagine a flow that will be triggered for each store for example (thousand of store, with 10+ tasks for each one), Airflow will not be able to manage this kind of workflow quickly (and it's not its goals). Airflow was clearly defined to handle small (hundreds tasks) for a long time.
For the XCOM part, Airflow store this in database, so you can't store data into this, you will need to store a small data (database is not here to store big files). In Kestra, we have a provide a storage that allow storing large data (Go, To, ...) between tasks natively with the pain on multiple node clusters.
AirFlow 2 was released in 2020. You're saying you knew that these issues were fixed, and then an article is published on your webpage in 2022 knowingly comparing against the technical properties of a major version release 2 years behind? That is not a good look.
First of all, the article published is a retrospective, we are talking from decision in 2019, we can't talk from the past that leed us for a choice?
Second, not all issues, some of them are fixed but there is still major issue, just dig google about issue scaling airflow on production, even with airflow 2, it's still complicated. Airflow still use a lot of CPU for doing nothing else than waiting for some api call. Just try to run 5000 tasks that sleep (simulation of an api call) in Airflow and we will see the challenge of scaling it.
Third, Airflow have still design issues that will not allow you to deal with some sort of pipeline.
Last one, I'm not here to fight against Airflow, some people love, some people hate it. We have take a completely different choice about designing and scaling data pipeline, I let people used what they like. For me, Airflow (and other workflow manager) doesn't fit.
I don't think it's correct to think of a laser as a source in some thermal equilibrium. "Concentrating temperature" passively from sources in thermal equilibrium is forbidden, but there's nothing preventing "concentrating power".
Pulsed lasers bring material interactions into a highly non-linear regime - photon intensity is so high that multiple photon absorption is common. In the typical nuclear decay regime you are concerned with single photon absorption, and the gamma ray intuition is correct. There are also a number of approaches where various targets hit with ultrafast lasers produce controllable flux of gamma rays which are used in downstream experimentation.
Have we found diseases that could propagate across the world either (1) with virality greatly in excess of modern diseases, i.e. R0 >> omicron or (2) untreatable/curable and potentially latent for weeks to months?
A disease capable of coming close to ending civilization would need to have properties far beyond any disease observed so far. Either it needs to infect massive populations before we detect it, or it has to transmit over long distances (miles) despite e.g. moderate precautions like masking, air filtering. I think there's good reason to doubt such a pathogen could exist. The closest I could imagine would be an HIV-like immunodeficiency virus that can be transmitted via aerosol - but even that would have to cause disease much more severe than HIV without resistance among even .01% of the population.
Obvious no-true-scotsman. Believing that the goal of crypto is to circumvent laws regarding possession and theft is at most a fringe belief. The fact that this is at the top of HN demonstrates how devoid of merit crypto discussion here is.
Circumventing property law has absolutely been a selling point of crypto, front and center, since its inception. The real no-true-Scotsman is saying, "Those who want an extra-governmental system of property aren't the real crypto fans." Yes, they are.
In reality, crypto's true purpose is a moving target, so it can never be criticized because that's not what crypto is really about.
I know too many people who aim to become millionaires off of btc and then move to countries without cap gains tax. I feel their sentiments are held by a large constituent.
The flipside viewpoint is that cryptocurrency bypasses censorability by large corporates; cryptocurrency embodies the freedoms that appear to be espoused so loudly and delightedy by "the US"^.
Guns = good
Cryptocurrency = bad
Opinions I agree with = good
Opinions I disagree with = bad
Me getting mine = good
Someone else getting theirs = bad
Censorship is the battleground issue for the 2020's.
^Apologies for the lumping of 300 million people into a single sentence description, it's for the sake of trying to make a point of the entanglement of "the US" and "freedom" - which isn't a bad thing.
Lots of tools that are useful for marginalized groups are also useful for marginalized groups that want to spread hate. Same goes for the internet. It was hard in the beginning on internet mainstream (late 90s/early 00s) to find any community that didn't have a bunch of racists and fascists in it as well.
And it's official, Godwin's Law [0] has reached crypto. The purpose of Bitcoin is of course to support the Nazi cause. What other purpose could it possibly have? /s
Hint: Millions of people use Bitcoin as a:
- store of value to protect purchasing power over time
- inflation hedge to protect savings from the ravages of inflation
- a hedge to protect against corrupt governments manipulating currency
- protection from negative real interest rates
- censorship-resistant payments
- anonymous payments with instant finality (Lightning)
Money is a tool like any other. Cash, gold, NFTs, Bitcoin, and credit cards can be used for good or evil, lawful or unlawful purposes. The technology isn't inherently moral or immoral. It is just a tool.
The "it's really for nazis" argument is particularly weak. The critics must be getting desperate.
This is the first time you hear "Bitcoin is for nazis"? You're lucky, I guess. It's a pretty common accusation because many Bitcoin advocates are also far-right and/or extreme libertarian.
Anyway, I'm not saying that. Eichmann is simply a reductio ad absurdum example of the problems with the "it's just a tool / technology has no moral" position.
Keeping the government from "seizing" assets is a big crypto goal, and avoiding taxes is included in that for a subset of users (which is "theft" if you agree with them). What to do with actual theft is kind of swept under the rug until it actually happens to a cryptobro, at which point they are very much in favor of a central authority getting their bitcoins or ugly monkey jpegs back.
That's such a sad view, especially to hold in a generally curious place like HN. Another example is narcotics, yes we know that most narcotics usage is bad, but does that mean all usage of narcotics is bad? Obviously not, and we take those articles as they come, and discuss the angles each article has independently, in most cases at least. But somehow cryptocurrencies are so emotional for most people, that they hold such a black/white view of it.
We can, and should discuss subjects without "tainting" them with general, over-discussed points when we can, especially if we want to keep HN curious and not turn into a echo-chamber.
It might take time, but the fact that the government can't print free bitcoins as it does with fiat to pay off its usurious debt and devalue everyone's hard work is a compelling basis.
But won't you just hoard all of your earnings if it isn't inflated away by 2+% every year? I've been told spending would grind to a halt. Also without holding your wealth in cash in a bank, how would banks use your money as a reserve for lending out to their favored clients?
Not necessarily. Secondly, banks and money lending are immoral and predatory. Usury is prohibited in the three major religions (Islam, Judaism, and Christianity), so we're better off without this dangerous practice. It goes hand in hand with fiat money by the way, the government is taking loans from the Fed, which is why it keeps needing to print more and more money to fuel it. The sooner we get rid of money lending as a business, the better.
On a side note, Islam requires a 2.5% Zakat from money hoarded in your account, to be donated to charity, so there's your solution against hoarding :) We don't need the government to fake print money to prevent people from hoarding. Better that money go into charity to truly have a more equitable society, as opposed to the fake and useless proposals we keep seeing and pitting parties against each other.
The Zakat is an interesting concept, but I don't see much functional difference from inflating the currency by 2.5% and then giving the newly created currency to the poor. Presumably some system is needed to enforce Zakat, that same mechanism of force could be used to inflate currency.
Since inflation is a centralized operation and Zakat is decentralized, I would wage enforcement of inflation is much easier than enforcement of Zakat.
According to https://www.usgovernmentspending.com/welfare_spending_analys..., in FY2021 welfare (not including Social Security or Medicare, which are for retirees, but including Medicaid) was $2,418B across federal, state, and local, about 76.6%. Neither Zakat or US welfare spending includes discretionary charity.
Overall, US welfare spending seems to be on the same order as, albeit a little less than, a Zakat imposed on all US wealth. Also, I'm not sure if this welfare figure includes EITC, which is the logical way that additional cash benefits should be distributed (since it avoids welfare cliffs).
Different types of wealth have different Zakat values, and not all wealth is subject to Zakat (e.g. the value of your home does not count towards Zakat). For example, currency, including fiat, gold, silver and other precious metals, is at 2.5% annually. Livestock has a different calculation, and so does produce.
Bitcoin and ETH combined are over 1T USD, much more than the figure you quoted. That's 25B annually, imagine how many lives that can change. Not to mention gold, which is at 11T, so 250B annually. Insane money that can revamp the entire planet.
It's strictly superior to have a system based on Zakat than the insane income taxes that we have today.
> Neither Zakat or US welfare spending includes discretionary charity.
Zakat is the bare minimum required for Muslims to pay per year. Islam heavily encourages discretionary charity, called Sadaqah. Both approaches are complementary.
> Different types of wealth have different Zakat values, and not all wealth is subject to Zakat (e.g. the value of your home does not count towards Zakat).
Of course. I did not want to get into such complications. This was more of a Fermi estimate to compare the amount a Zakat would raise in the US.
> It's strictly superior to have a system based on Zakat than the insane income taxes that we have today.
Maybe - remember that the US government pays for more besides bare welfare for the needy. Also the Islamic Zakat pays for more than welfare - also administration of Zakat (reasonable, but should be kept as low as possible) and Islamic missionary efforts (I don't think the US should redirect its welfare to "spreading liberty and democracy").
Even in Islam, there were more taxes than Zakat[1] - at the very least, a tax on harvests (corporate income or business reciepts tax) and a land tax - because Islamic governments also have other responsibilities besides charity. It would stand to reason that the federal and state governments would also continue to collect other taxes to support other government responsibilities. Also remember that inflation (certainly that intentionally engineered by the central bank) is effectively a wealth tax.
Which missionary efforts? If you mean paying Zakat to those whose hearts are inclined toward Islam that's something different.
> a tax on harvests
I mentioned this in my previous post. Livestock and produce have different Zakat calculations than the 2.5% of money held for a year.
If you're referring to Ushr, that's imposed on non-Muslim nations that taxed Muslims, so a tit-for-tat treatment, and it's not part of Islam per-se, but a socio-political decision.
> certainly that intentionally engineered by the central bank
Exactly what we don't want. We don't want a select few people to determine the tax rate for the entire population, affecting mainly people at the lower socioeconomic levels in society.
I'm going to assume you're from one of the countries mentioned below? It's my understanding most countries with Muslim majority do not centrally enforce Zakat.
>Today, in most Muslim-majority countries, zakat contributions are voluntary, while in Libya, Malaysia, Pakistan, Saudi Arabia, Sudan, and Yemen, zakat is mandated and collected by the state (as of 2015).[16][17]
[wikipedia ^]
>How so? Could you elaborate?
By compelling people to hold wealth denominated in currency, and then inflate that currency using central bank or treasury.
> It's very different because when the government inflates the currency, we all know whose pockets it ends up going into :)
No disagreement here. But government can also misappropriate Zakat. I am actually not in favor of most forms of planned inflation nor a compulsory Zakat precisely in part because I predict massive fraud on the minority of those holding the power to distribute it.
> I'm going to assume you're from one of the countries mentioned below? It's my understanding most countries with Muslim majority do not centrally enforce Zakat.
Today, many Muslim countries are not in a good shape unfortunately, and that's due to several reasons beyond the scope here. I'm referring to how things are required by Islam, and how things were done historically when Islam was actually implemented. Today Islam is not applied 100% unfortunately, which is a main cause of weakness for Muslim nations. They're either in property due to occupation, current or historic, or have to bow down to the West's whims, so that they are not overturned or invaded.
> By compelling people to hold wealth denominated in currency, and then inflate that currency using central bank or treasury.
I meant how would inflation happen in case of Zakat being enforced? They're orthogonal things unless I'm misunderstanding you.
> But government can also misappropriate Zakat
There is a set category of people whom are deserving of Zakat clearly outlined in the Quran. So, if Islam is properly applied, there would not be any meddling. It has been historically documented that in Iraq during Ummayad rule, there were no more poor people left to accept Zakat. Quite amazing.
You've said If Islam is properly applied, there would not be any meddling and Zakat would be applied appropriately. I guess you have more faith than I do that a human vested with these large sums will apply it appropriately. Humans can be greedy, corrupt, and clumsy and may not practice the word of the Quran accurately. They could simply make bad mistakes, without any bad will. Having centralized access to large sums of alms could result in Zakat going to benefit other powerful parties as well. Since it is collected by government, it would be hard for those paying it to stop payment in protest if someone did start meddling. Note this is also a problem with secular welfare systems, I am not saying it is a problem only with centralized Zakat.
>It has been historically documented that in Iraq during Ummayad rule, there were no more poor people left to accept Zakat.
Umayyad had a variety of religions of persons overseen by their caliphate. Was Zakat distributed to poor Christians in the Caliphate? Or did Zakat only go to Muslims? This is important to know, because supporting only minority of the poor who practice Islam could mean Zakat may have not solved poverty for the entire populace. I'd also be interested in seeing the citation that poverty didn't exist under this caliphate.
In America we have the problem that lots of administrators and bureaucrats siphon off much of the money in the welfare system into their salaries as well as issues with the money going into the hands it is intended to go to. Also since the money is taken by force, there's not a lot of control by those who contribute the money over making sure it is used appropriately.
I will note I find it both fascinating and worthy of respect that many cultures have come up with their own ways of helping the poor.
Personally I would be much more on board with a decentralized type of Zakat where individuals can pick what charity to go to, in order to protect from centralized failures of government.
>I meant how would inflation happen in case of Zakat being enforced? They're orthogonal things unless I'm misunderstanding you.
If I wanted to enforce Zakat via inflation, I would mandate people to hold their money in bank, as debt, or fiat denominated bonds, and then I would inflate the money supply by 2.5% by mailing out 2.5% of the current supply to the poor per year, or something approximating that. But that would also be an imperfect system.
> I guess you have more faith than I do that a human vested with these large sums will apply it appropriately.
I agree that humans can be greedy, etc. But that's why we have a judicial system. When you look at history, the Islamic scholars took their faith extremely seriously. Their accounts and biographies are not something you'd find in Western texts, but those exceptional people truly did exist. And because of them, we had things like the Islamic Golden Age.
The governments today have way more money on their hands don't they? Especially with the insane taxation rates we see. You allude to this point when you mention secular welfare systems. But history shows otherwise when Islam was applied.
> Umayyad had a variety of religions of persons overseen by their caliphate
Islam was the dominant religion, and the majority of the population were Muslims. The non-Muslims had to pay Jizya (limited to able men, i.e. not women, children, old men, or religious priests).
One of the categories of people who are eligible to receive Zakat, are those whose hearts are inclined toward Islam. Other than that, I don't think non-Muslims receive it. That being said, poor and needy non-Muslims are definitely eligible for charity (Sadaqah), and it is the responsibility of a functioning government to ensure that its population is well taken care of. Islam guarantees the rights of non-Muslims, and is very strict about it.
> I'd also be interested in seeing the citation that poverty didn't exist under this caliphate.
I didn't claim that no poverty existed in the entire Caliphate. As you know, the Caliphate spanned several regions and districts. I mentioned the Iraqi district, but I came across this question[1], which mentions that the mayors of the Libiyan and Tunisian regions wrote to Umar ibn AbdulAziz that they could not find a needy person to give them Zakat, so he responded to give it to the poor among the Jews and Christians. They replied that still no one took it, and they were no needy among them, so Umar replied to leave it in the market for anyone to take as they need. When still no one took it, Umar ordered to purchase slaves and free them.
I'll have to validate the authenticity of this specific account, but the notion that during Umar's rule, in certain districts there were no poor people left to accept Zakat is established.
> If I wanted to enforce Zakat via inflation, I would mandate people to hold their money in bank
Ah I see. It's prohibited in Islam to hold someone's money against their will, so there goes that :)
>The governments today have way more money on their hands don't they? Especially with the insane taxation rates we see. You allude to this point when you mention secular welfare systems. But history shows otherwise when Islam was applied.
Yes and I think the level of money and trust the government holds is a source of massive failure. I think those practicing Islam should be free to distribute their Zakat directly to poor or their select organizations that aid poor, rather than being forced to give it to a central authority. Otherwise one central authority has a monopoly on distribution of Zakat, which can lead to many inefficiencies and failures.
>One of the categories of people who are eligible to receive Zakat, are those whose hearts are inclined toward Islam. Other than that, I don't think non-Muslims receive it. That being said, poor and needy non-Muslims are definitely eligible for charity (Sadaqah), and it is the responsibility of a functioning government to ensure that its population is well taken care of. Islam guarantees the rights of non-Muslims, and is very strict about it.
Again I think it's wonderful that people are offered this kind of charity. I'm a little skeptical that the system under the caliphate could have prevented all poverty or that the poverty that remained wasn't solved by collecting Jizya from non-muslims and then distributing Zakat only to those whose hearts are inclined toward Islam. I admit I do not understand much of the history of Muslim Caliphates or nations, so I'm unable to really ascertain where islamqa.info gets its source from, but I doubt we have very good record of income distributions under this caliphate. But hey, I don't have any proof that there were poor, so maybe it's true.
I do thank you for digging up your source in this matter, and it is interesting to note some points on records of history.
> Ah I see. It's prohibited in Islam to hold someone's money against their will, so there goes that :)
A reasonable prohibition, one I extend to involuntary taxes and forced centrally collected charity.
Thank you for your viewpoint here, as it's one I rarely see living in the west.
Narcotic may be whatever you want, but the debate has to be honest and transparent, not shady and criminal: if you want to legalize heroin, make a case to the people, get it approved with safeguards, say like Portugal or France to some extent (Methadone), and make sure it won't profit people who also do other things with the proceeds of sales.
The problem with Narcotic is simple: people lose freedom when getting into addiction, then with hard narcotics they also lose the ability to make important life choices, minors and generally disfavored people are targeted by addictive substance sales people, production is rarely done well (because most fields should be used for food, if they are used for more profitable narcotic purpose legally or due to lax enforcement, ALL FIELDS become opium fields like in Afghanistan, which can cause food safety issues etc.
You can't just talk as if narcotic consumers are innocent party goers. Many are absolute victims and we must talk about it without crypto entering the debate. Crypto is just a weak way to try to hide the source of completely illegal funding for narcotics without going through the painful discussion with the population that we may have to sacrifice a lot of victims for the sake of spending less on narcotic enforcement.
I think in mechanical metamaterials the characteristic length defining the "metamaterial region" is rather the wavelength of pressure waves in the material you're considering - much like in electromagnetics you want the patterning (cell) length to be much less than the wavelength of radiation. In work like this they are effectively looking at 0.1 Hz or lower - near static loading - so I think pattern size can be quite large (around 600 m wavelength in bulk rubber for 0.1 Hz). This interpretation also replicates the localized behavior in the shock experiment videos. When the platform is dropped an impulse is applied with frequencies above the metamaterial regime for the material, so you see highly asymmetric response through the material - implying that the macroscopic "metamaterial" property characterization is insufficient to predict response, and so analysis must be done at feature scales rather than wavelength scales. The idea being that a "metamaterial" is a structure that can be treated as a bulk continuous material with a particular defined response as long as the interacting frequencies are all sufficiently low (far below the characteristic wavelength of the material).
I think the bending analysis you cite can determine the relative feature sizes desirable for certain "micro-scale" mechanical behavior, but it's possible to build a mechanical "metamaterial" much larger than that as well.
The sample distribution of viruses is incredibly important for this sort of analysis, and much of the argument here only makes sense through the lens of uniform virus sequencing. If you have imbalanced sequencing and imbalanced transmission you can also explain these differences.
The important thing is that mutations occur at a certain rate per virus per unit time. If you have an isolated population that's sequenced infrequently then (1) that strain will appear to evolve more slowly as there's a smaller population capable of mutating, and (2) once that strain is sequenced it's going to look far from what you've seen already since you haven't been tracking the intermediate mutations in this population.
The S/N ratio can be analyzed in terms of a random walk in high dimension. Variance in these walks grows over time (in terms of distance from origin, i.e. number of mutations), so the discrepancy doesn't seem super far from what's plausible under the null hypothesis. Perhaps someone can do the math on that.
The hypothesis merits further investigation, but the strength of the evidence presented here really requires some complex statistical analysis to determine if innocuous explanations fit. The analysis is far more complex than I would expect an epidemiologist or virologist to apply in the course of their work.
Mutation is tricky, because basically the only way to do it sanely is to copy all data into the residual, but since most arrays aren't actually mutated, that's extremely wasteful. I've been hoping to address this by changing mutability in the language more generally (e.g. https://github.com/JuliaLang/julia/pull/42465) to make immutable arrays the default at which point there wouldn't be a penalty anymore. I've had request to do the mutable copying optionally, but it's a bit tricky, because it needs rule system integration and the rule system currently doesn't reason about mutation.
As for exceptions and recursion, shouldn't be a problem, just needs to be implemented.
Also I note the only thing you have posted before is a link to this paper in particular.